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ABSTRACT 


With  the  explosion  of  available  data  from  a  variety  of 
sources,  it  has  become  increasingly  difficult  to  keep  pace 
with  the  amount  of  arriving  data,  extract  actionable 
information,  and  integrate  it  with  prior  knowledge.  Add  to 
that  the  pressures  of  today's  fusion  center  climate  and  it 
becomes  clear  that  analysts,  police  officers,  and 
executives'  ability  to  make  rapid,  sound  decisions  is 
severely  compromised. 

The  combination  of  circular  reporting  and  the 
complexity  of  data  sources  are  contributing  to  information 
overload.  Law  enforcement  agencies  realize  this  cannot  be 
resolved  by  continuing  to  hire  more  intelligence  analysts. 
Instead,  they  must  begin  leveraging  technology.  The 
Illinois  State  Police  is  utilizing  a  technology  artifact  in 
its  Statewide  Terrorism  and  Intelligence  Center  (STIC)  which 
incorporates  technology  built  from  the  context  of  these 
users.  This  thesis  uses  a  survey  instrument  to  evaluate  the 
effectiveness  of  this  technology  on  reducing  circular 
reporting  and  the  handling  of  complex  data  sources. 

The  findings  conclude  that  intelligence  analysts  within 
STIC  perceive  that  information  overload  exists,  and  both  the 
complexity  of  data  sources  as  well  as  circular  reporting 
minimizes  their  effectiveness  and  efficiency.  Furthermore, 
this  technology  effectively  improves  these  negative  effects, 
and  increases  the  STIC' s  ability  to  better  serve 
communities . 
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I .  INTRODUCTION 

A.  GENERAL  BACKGROUND 

In  recent  years,  rapid  advances  in  the  field  of 
information  and  communication  technologies  have  led  to  the 
emergence  of  new  technology  that  is  transforming  the  world 
into  a  data  driven  society.  Unstructured  text  in  the  form 
of  emails,  web  pages,  and  word  processing  documents  has 
become  overwhelming  and  has  created  significant  obstacles  in 
extracting  meaningful,  actionable  information  used  for 
decision-making . 

Today' s  fusion  centers  are  faced  with  an  urgent  need  to 
efficiently  and  effectively  build  on  prior  information 
providing  analysts,  police  officers  and  executives  the 
ability  to  make  timely  and  accurate  decisions.  The  primary 
barrier  to  achieving  this  goal  is  effectively  managing 
information  overload. 

The  problem  with  most  technological  solutions  to  the 
information  overload  problem  is  that  the  contributing  factor 
cannot  be  clearly  mapped  to  a  particular  process.  Rather, 
it  is  manifested,  in  the  large  collection  of  interdependent 
processes  that  are  inherently  related  to  a  person's  tasks, 
information  needs,  environment,  and  role. 

Most  technology  solutions  focus  on  a  particular  process 
and  do  not  consider  the  interdependencies  between  processes 
that  create  these  problems.  No  one  specific  solution  to 
this  information  overload  problem  exists,  nor  does  just  one 
problem  exist. 
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This  thesis  will  focus  on  the  information  overload 
problem  that  exists  within  state  intelligence  fusion 
centers,  including  specifics  about  how  these  problems 
manifest  at  the  Illinois  State  Police  (ISP)  Statewide 
Terrorism  and  Intelligence  Center  (STIC) .  These  problems 
are  described  as  "Circular  Reporting"  and  "Complexity  of 
Data  Sources." 

B.  STATEMENT  OF  THE  PROBLEM 

Along  with  the  explosion  of  available  data  from  the 
Internet,  news  feeds,  internal  data  sources,  sensors  and 
other  emerging  forms  of  data,  it  has  become  increasingly 
difficult  to  keep  pace  with  the  amount  of  arriving  data, 
extract  actionable  information,  and  integrate  it  with  prior 
knowledge  (manually) .  Add  to  that  the  pressures  of  today' s 
fusion  center  climate  —multi-tasking,  long  hours,  limited 
resources,  mental  fatigue—  and  it  becomes  clear  that 
analysts,  police  officers  and  executives'  ability  to  make 
rapid,  sound  decisions  is  severely  compromised.  Decision¬ 
makers  faced  with  this  blitz  of  information  know  that  in 
order  to  make  timely  and  effective  decisions,  they  must 
analyze  and  respond  to  information  quickly. 

Both  law  enforcement  and  the  business  sector  have  been 
challenged  to  keep  pace  with  the  amount  of  information  that 
is  being  made  available  to  them.  Illinois  State  Police 
Senior  Intelligence  Advisor  Aaron  Kustermann  estimates  that 
"65-80  percent  of  analysts'  time  is  spent  gathering  and 
collating  data  with  only  a  small  percentage  of  time 
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remaining  for  true  analysis".1  This  compromises  the 
decision-making  process  at  both  the  executive  and  tactical 
intelligence  levels,  putting  in  jeopardy  our  ability  to 
accurately  conduct  long-range,  strategic  planning  and 
proactively  address  crime.  Further  worsening  the  problem  is 
the  fact  that  budgets  available  for  staffing  increases 
remain  flat,  while  the  amount  of  information  available 
continues  to  increase  exponentially. 

The  media  is  rife  with  reports  and  complaints  about 
information  overload.  It  is  viewed  as  a  symptom  of  our 
technologically  savvy  society,  and  is  pervasive,  affecting 
people  in  every  occupation.  Attempts  to  deal  with 
information  overload  range  from  advice  about  work  habits  to 
time  management  to  stress  relief  to  the  application  of  more 
technology.  All  too  often  the  latter  not  only  does  not 
solve  the  problem,  it  makes  it  worse.  In  particular,  many 
technological  approaches  to  solutions  simply  provide  one 
more  place  where  you  have  to  go  for  information,  provide  yet 
one  more  report  you  have  to  read,  and  so  on. 

The  problem  with  most  technological  approaches  to  the 
information  overload  problem  is  that  the  contributing  factor 
is  not  just  one  information  problem.  The  "problem" 
manifests  itself  in  a  variety  of  ways,  and  is  actually  a 
large  collection  of  problems,  which  are  inherently  related 
to  a  person's  tasks,  information  needs,  environment,  and 
role  in  his  or  her  organization.  The  following  guotation 
from  BulletinNEWS, 


1  Aaron  Kustermann,  (Senior  Intelligence  Advisor,  Illinois  State 
Police)  interview  by  author,  November  17,  2006. 
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Overload.  When  your  executives  receive  too  much 
information,  they  either  waste  significant  time 
wading  through  it  all,  or  worse,  ignore  it 
altogether.  That  ignorance  and  resultant  loss  of 
competitive  edge  can  be  very  costly  when  it's 
time  to  make  important  decisions.  Clipping 
services  do  little  to  solve  this  problem,  and 
arguably  make  it  worse.  A  newspaper  article 
provided  as  part  of  a  clipping  service  might 
include  a  piece  of  information  valuable  to  the 
company  only  in  its  17th  paragraph,  but  each 
executive  of  this  company  is  forced  to  find  this 
nugget  by  reading  the  whole  story,  much  of  which 
is  valueless.2 


shows  one  example  of  this;  the  use  of  the  technology-based 
news  clipping  service  still  does  not  solve  the  problem  for 
the  harried  executive  or  for  his  organization.  There  is 
still  too  much  time  wasted  in  reading  and  searching  for 
"nuggets."  What  is  missing  in  the  solution  is  some 
contextual  knowledge  of  what  constitutes  a  "nugget" 
(relevance  to  mission,  role,  and  task)  and  a  design  of  a 
solution  that  reflects  personal  and  organizational  workflow 
and  time  management  needs . 


To  note  a  specific  manifestation  of  an  information 
overload  problem  one  might  use  the  cliche,  "the  devil  is  in 
the  details."  In  order  to  deal  with  those  details,  a 
particular  solution  must  be  constructed  to  address  the 
situation,  as  well  as  a  contextual  and  task-specific 
understanding  of  the  specifics.  No  one  specific  solution  to 
this  "information  overload"  problem  exists,  as  not  just  one 
problem  exists.  Information  Overload  problems  occur  in 
every  conceivable  variety  of  contexts,  and  cannot  be  solved 


2  "Corporate  News  Briefings."  BulletinNEWS , 
http://www.bulletinnews.com/corporate  briefings.htm  (accessed  December 
2,  2007) 
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with  a  single  solution;  by  way  of  analogy,  consider  a 
problem  of  "Faster  Transportation."  A  solution  to  providing 
"Faster  Transportation"  could  involve  anything  from  an  F/A- 
18  Hornet  to  a  C5  Galaxy  to  a  Virginia-class  submarine  to  an 
XM1108  Universal  Carrier  to  a  V150  TGV  high-speed  train  to  a 
Bugatti  Veyron  automobile  to  a  Lightning  F-40  bicycle.  The 
"right"  solution  can  only  be  determined  by  examining  and 
understanding  the  contextual  and  task  specific  needs.  There 
is  no  single  "Information  Overload"  problem  anymore  than 
there  is  a  single  "Faster  Transportation"  problem. 

Examples  have  been  revealed  throughout  the  literature 
suggesting  other  agencies  are  struggling  with  similar 
challenges.  Further  evidence  in  support  of  this  issue  can 
be  found  in  Kelderman' s  report,  "Federal,  state  and  local 
law  enforcement  agencies  are  building  or  improving 
information-sharing  programs  to  prevent  terrorist  attacks 
and  improve  police  work.  A  process  that  trades  stronger 
vigilance  for  complaints  of  information  overload."3 

Kelderman  describes  a  situation  in  which  a  naive 
approach  that  constructs  a  solution  to  information  sharing 
using  technology  without  considering  the  other  contextual 
factors  necessary  for  crafting  an  effective  solution  can,  in 
fact,  make  things  worse.  Important  national  strategic 
objectives  are  being  hindered  by  neglecting  these  other 
crucial  factors  when  creating  specific  contextually 
sensitive  solutions. 


3  Eric  Kelderman,  "Law  Enforcement  Linking  Tangled  Information, " 
Stateline . org, 

http : / /www .stateline . org/live/ViewPage . action?siteNodeId=136&languageId= 
l&contentld=15500  (accessed  January  31,  2007). 
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At  the  Federal  level,  we  find  further  evidence  of  the 
issues  caused  by  information  overload.  For  instance,  "A  big 
part  of  the  problem  is  that  intelligence  agencies,  including 
the  FBI,  simply  collect  too  much  information  and  can't  sift 
through  it  effectively.  Figuring  out  how  to  deal  with 
information  overload  is  a  key  management  challenge."4 

Enterprises  at  all  levels  of  government  and  the 
private  sector  are  creating,  collecting,  and 
storing  more  data  than  ever  before.  The  trend 
extends  across  the  globe.  According  to  a  U.C.- 
Berkeley  estimate,  more  than  five  exabytes  of  new 
information  (1018  bytes)  are  created  every  year 
worldwide  -  enough  to  fill  37,000  new  Libraries 
of  Congress  and  more  than  all 
the  words  ever  spoken  or  printed.  This  is  the 
annual  eguivalent  of  a  30  foot  stack  of  books  for 
every  man,  woman,  and  child  on  the  planet.5 

Following  the  tragic  events  of  9/11  the  nation'’ s 
leaders  insisted  those  responsible  for  public  safety  share 
information.  Little  doubt  exists  that  those  entities  are 
sharing  more  data. 

Perhaps  nowhere  is  unstructured  data  piling  up 
faster  than  at  American  security  organizations 
tasked  with  winning  the  war  on  terrorism.  They 
are  inundated  with  an  unprecedented  'volume, 
velocity  and  variety, '  or  '3Vs, '  of  data,  causing 
massive  'information  overload"  According  to 
congressional  testimony  and  other  public  sources. 


4  Brian  Friel,  "Behind  the  Badge,"  GovExec.Com  (December  1,  2001) . 
http://www.govexec.com/features/1201/1201s3.htm  (accessed  April  20, 

2007)  . 

5  Scott  Greiper  and  Mark  Sauter,  The  Business  of  Connecting  Dots:  The 
$1  Billion  Intelligence  and  Security  Inf ormatics/ Analytics  Market 
(Annapolis,  MD :  C.E.  Unterberg,  Towbin  and  Chesapeake  Innovation  Center, 
[2005 ]) ,  4 .  http : / /www . aatechcouncil . org/ documents/ Connect ingDots . pdf 
(accessed  March  17,  2007) . 
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U.S.  security  agencies  are  collecting  far  more 

information  than  they  are  able  to  process.6 

It  is  incumbent  upon  the  leaders  in  Homeland  security 
to  find  a  solution  to  the  information  overload  problem  that 
has  emerged.  In  what  follows,  we  will  explore  some  of  the 
specific  ways  that  information  overload  problems  manifest 
themselves  at  the  Illinois  State  Police. 

Beginning  in  early  2000  and  continuing  through  2002 
many  metro-St.  Louis,  Missouri,  law  enforcement  agencies 
were  independently  investigating  dozens  of  murders.  These 
murders  had  many  similarities,  but  the  majority  of  the  data 
had  never  been  analyzed.  Once  analysis  began,  certain 
important  patterns  developed  such  as  sex,  race,  occupation, 
body  concealment,  last  seen  location,  etc.;  but  there  were 
other  large  amounts  of  information  left  unanalyzed, 
including  but  not  limited  to,  thousands  of  reports  written 
in  several  formats  by  numerous  departments.  Those  involved 
in  the  analysis  determined  that  this  constant  information 
flow  would  guickly  overwhelm  them. 

At  the  same  time  in  central  Illinois,  the  ISP  was 
preparing  to  open  the  Statewide  Terrorism  Intelligence 
Center.  The  ISP  realized  the  overwhelming  responsibility  of 
being  accountable  for  the  data  contained  in  file  cabinets, 
dozens  of  desperate  databases,  and  the  memories  of 
employees . 

During  conversations  with  policing  professionals  of  all 
ranks  and  from  all  types  of  agencies  it  became  apparent  that 

6  Greiper  and  Sauter,  The  Business  of  Connecting  Dots:  The  $1  Billion 
Intelligence  and  Security  Inf ormatics/ Analytics  Market,  5. 
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each  of  them  felt  they  were  overloaded  with  data.  The 
practice  known  as  "circular  reporting"  is  filling  in-boxes 
across  the  country;  street  officers,  analysts  and  executives 
fear  deleting  the  one  critical  piece  of  information  that  may 
make  the  difference.  It  is  commonplace  for  us  to  read  the 
same  information  numerous  times,  reducing  available  time  to 
focus  on  relevant  issues. 

C.  INFORMATION  OVERLOAD  EXPLAINED 

1 .  Circular  Reporting 

Classic  circular  reporting  occurs  when  two  or  more 
sources  report  the  same  information  to  an  intelligence 
agency,  although  it  in  fact  originated  from  the  same 
source . 7 


intel  agency 


Figure  1.  Classic  Circular  Reporting 


If  law  enforcement 
intelligence  reports, 
will  have  the  same 
information  at  all. 


officers  are  deluged  with 
the  information  overload 
outcome  as  not  sharing 
If  officers  are  deleting 


7  Kyle  Teamey  and  Jonathan  Sweet,  "Organizing  Intelligence  for 
Counterinsurgency,"  Military  Review  86,  no.  5  ( September-October  2006) : 
28.  http : //usacac . army .mil/ CAC /mil review/ English/ SepOct06/Teamey- 
Sweet.pdf  (accessed  January  15,  2007). 
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intelligence  products  without  reading  them,  then 
the  effect  is  the  same  as  if  it  had  never  been 
disseminated. 8 

Another  key  element  to  the  circular  reporting  problem 
is,  of  course,  the  duplication  that  exists  among  the 
multiple  reports.  Compounding  the  issue  is  that  not  all 
reports  are  completely  identical.  In  the  absence  of  a 
detailed  reading  of  each  incoming  item  and  laborious  note 
taking,  it  is  nearly  impossible  to  reliably  extract 
meaningful  sources  of  intelligence.  Many  of  the  reports 
contain  information  that  originated  as  open  source 
intelligence  (OSINT) . 

The  classic  circular  reporting  problem  happens  when  two 
or  more  sources  report  the  same  information  to  an 
intelligence  agency,  although  it  in  fact  originated  from  the 
same  source9  (see  Figure  1) .  This  not  only  leads  to 
duplication,  but  it  can  over  emphasize  the  importance  of  a 
piece  of  information,  as  the  information  provenance  may  not 
be  readily  available,  i.e.,  it  may  not  be  apparent  that  the 
two  or  more  reports  originated  from  the  same  source. 

The  most  obvious  example  of  circular  reporting  at  the 
Statewide  Terrorism  and  Intelligence  Center  (STIC)  of  the 
Illinois  State  Police  (ISP)  is  that  of  the  Daily 
Intelligence  Reports  received  at  the  STIC.  This  also 
extends  to  the  weekly  and  other  periodic  reports  received 


°  David  L.  Carter,  Law  Enforcement  Intelligence:  A  Guide  for  State, 
Local,  and  Tribal  Law  Enforcement  Agencies  ([Washington,  DC]:  U.S.  Dept, 
of  Justice,  Office  of  Community  Oriented  Policing  Services,  2004),  86. 
http : //www . cops . usdoj . gov/mime/open . pdf ?Item=1439  (accessed  January  20, 
2007)  . 

9  Teamey  and  Sweet,  Organizing  Intelligence  for  Counterinsurgency, 

28. 
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from  various  law  enforcement  and  intelligence  agencies.  In 
many  cases  the  Daily  Reports  contain  similar  entries  in 
regard  to  recent  news  events  -  events  which  are  often 
already  in  the  public  domain  in  the  form  of  an  uncovered 
terror  plot,  a  threat  to  a  specific  target,  etc.  In  some 
cases,  redundant  information  was  previously  brought  to  the 
analysts'  attention  via  television,  newspaper  (print  and  on¬ 
line)  ,  or  other  media  sources.  Analysts  and  supervisors  at 
the  STIC  have  reported  that  the  Daily  Intelligence 
Bulletins,  while  containing  valued  intelligence,  are  not 
thoroughly  read  by  each  analyst. 

Due  to  time  constraints  and  the  number  of  Daily 
Reports,  analysts  often  focus  on  just  one  or  two  from  a 
source  (such  as  Chicago  JTTF,  Virginia  State  Police, 
Massachusetts  State  Police,  etc.)  that  is  familiar  to  them 
based  on  a  prior  experience  or  success  with  it.  This 
phenomenon  of  selectively  choosing  certain  reports  to  rely 
on  has  been  observed  at  the  STIC  as  evidence  of  the 
following  example. 

A  common  issue  experienced  with  the  Daily  Reports,  and 
in  fact  any  intelligence  document  stored  on  the  STIC' s 
network  repository,  is  the  inherent  difficulty  of  locating  a 
valued  document  or  specific  portion  of  a  document  once 
stored  in  one  of  the  hundreds  of  folders  within  the  network 
repository.  The  issue  of  specific  relevance  and  extreme 
difficulty  in  tracking  that  topic  once  stored  is  common.  In 
one  situation  a  supervisor  requested  information  on  a 
specific  topic  knowing  he  had  earlier  viewed  it  in  an 
intelligence  bulletin.  Five  analysts  were  asked  to  look  for 
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this  information  -  they  returned  five  different  documents 
obtained  from  five  different  areas  of  the  network 
repository . 

One  of  the  key  elements  to  this  "circular  reporting 
problem, "  as  experienced  by  the  STIC,  is  the  enormous  total 
volume  of  reports,  representing  an  information  flow  so  vast 
it  is  impossible  for  the  appropriate  analysts  to  have  the 
time  to  actually  read  much  less  comprehend  and  prioritize 
the  reports.  The  Illinois  State  Police  is  not  alone;  the 
Federal  Government  is  also  recognizing  the  increase  in  the 
number  of  reports  that  should  be  read.  "Collecting  more 
information  from  more  sources  will  require  more  federal 
analytical  capability  to  prevent  information  overload."10 

Another  key  element  to  the  circular  reporting  problem 
is,  of  course,  the  duplication  that  exists  among  the 
multiple  reports.  Compounding  the  issue  is  that  not  all 
reports  are  completely  identical.  Some  of  the  reports  may 
include  new  information  that  was  developed  or  became  known 
later,  while  others  may  include  value-added  information 
supplied  by  a  particular  agency  or  published  reports  with 
special  knowledge  (see  Fig.  2)  . 


10  James  B.  Steinberg,  Mary  Graham  and  Andrew  Eggers,  Building 
Intelligence  to  Fight  Terrorism  (Washington,  DC:  Brookings  Institution, 
September  2003),  5.  http://www.brookings.org/comm/policybriefs/pbl25.pdf 
(accessed  February  15,  2007)  . 
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ISP  STIC 


Figure  2.  ISP  Circular  Reporting  -  Intel  Agencies 

In  the  absence  of  a  detailed  reading  of  each  incoming 
item  and  laborious  note  taking,  it  is  nearly  impossible  to 
reliably  extract  meaningful  sources  of  intelligence,  some  of 
which  may  come  across  as  incidental  from  one  source  but  a 
major  discovery  when  applied  to  another.  Many  of  the 
reports  contain  information  that  originated  as  open  source 
intelligence  (OSINT) .  For  example,  information  may  be 
reported  from  a  variety  of  news  sources  (wire  services, 
newspaper  websites,  TV  network  websites,  and  copies  via 
intelligence  agencies)  and  additional  information  may  flow 
in  later,  which  may  or  may  not  contain  new,  updated 
information.  Various  intelligence  agencies  may  themselves 
pick  up  these  reports  and  supplement  them  with  additional 
information  available  on  that  topic  from  within  their  agency 
(see  Fig.  3  ISP  Circular  Reporting  -  Open  Source  Intel 
(OSINT)  Flows) . 
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Several  analysts 

at 

the 

STIC 

have  described 

the 

challenges  they  deal 

with 

to 

keep 

up  with  the  pace 

of 

incoming  intelligence 

sources . 

To 

describe  a  few,  some 

aspects  are: 

The  reports  are  organized  by  reporting  agency  (rather 
than  topic)  so  to  monitor  "your  topics"  requires  sifting 
efficiently  through  all  reports.  Analysts  tend  to 

specialize  in  particular  areas,  e.g.,  one  analyst  may 
specialize  in  domestic  hate  groups,  while  another  may 
specialize  in  Islamic  terrorist  groups;  one  analyst  may  be 
the  local  expert  in  bio-weapons  and  agro  terrorism,  while 
another  focuses  on  violent  crime,  weapons,  and  the  'E'  in 
CBRNE  methods  (Chemical,  Biological,  Radiological,  Nuclear, 
high  Explosive) . 

Some  people  (supervisors,  other  management)  must  be 
aware  of  all  topics,  so  it  is  extremely  difficult  to  keep 
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track  of  all  of  the  issues.  These  people  must  try  to  read 
all  the  reports,  and  mentally  audit  what  is  new,  what  is 
important,  what  is  redundant,  and  what  is  novel  or  unique. 

The  large  number  of  events  reported  makes  it  impossible 
to  read  about  all  of  them.  No  effective  means  exists  to 
help  determine  which  are  most  important  and  relevant  for  the 
analyst.  For  instance,  a  former  supervisor  at  STIC 
explained  that  when  he  first  started  at  STIC,  he  diligently 
tried  to  read  all  the  reports  on  the  topic  of  explosions  but 
even  with  concentrating  primarily  on  that  topic  he  was 
quickly  overcome  with  information  as  the  number  of 
explosions  occurring  everyday  in  Illinois  alone  were  too 
much  to  keep  up  with.  Early  efforts  at  information  sharing 
have,  in  fact,  made  the  problem  worse  as  the  ease  of 
reporting  and  sharing  has  inflated  the  volume  of  reports 
significantly  and  thereby  produced  redundant  information. 
Many  reports  have  become  equivalent  to  clipping  services  as 
they  attempt  to  put  filters  on  information  to  gather  the 
most  essential  elements  of  events.  The  unfortunate 
consequence  of  this  improvement  is  distinguishing  redundancy 
from  equivalency  and  equivalency  from  something  unique  among 
multiple  sources.  As  analysts  work,  their  biggest  fear  is 
overlooking  a  unique  component. 

The  notes  and  annotations  they  wish  to  memorialize, 
must  be  dealt  with  manually,  which  further  complicates 
indexing  information  that  analysts  have  already  read. 
Easily  locating  information  (either  current  or  past)  is 
difficult  with  the  ISP's  current  setup.  Reports  often  come 
in  email  and  are  placed  onto  a  shared  network  drive  to 
facilitate  access  by  multiple  analysts.  Many  events  are 
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"cut-and-pasted"  from  reports  by  analysts  into  their  own 
electronic  filing  system  (usually  Word  documents  in  folders 
they  have  organized  themselves) . 

The  quantity  and  complexity  of  information  also  makes 
it  difficult  to  determine  crime  patterns  between  reported 
events,  for  example,  do  patterns  or  similarities  exist 
between  types  of  explosions,  e.g.,  ingredients,  locations, 
etc . 

2 .  Complexity  of  Data  Sources 

Many  cases  or  even  topics  of  interest  require  extensive 
document  and  database  collections.  These  efforts  often 
extend  over  long  periods  of  time  and  involve  many 
individuals . 

A  shortcoming  of  current  processes  are  effective  tools 
that  provide  both  great  detail  at  the  most  granular  level, 
as  well  as  tools  that  provide  an  overview  of  the  data.  In 
addition,  other  state  agencies  and  municipal  law  enforcement 
entities  have  access  to  their  own  unique  and  valuable  data 
sources . 

Increasingly,  government  and  corporate  customers 
are  realizing  that  without  innovative  technology 
solutions,  all  these  data  sets  will  simply  create 
'information  overload, '  at  best  wasting  storage 
space  and  at  worst  clogging  the  decision  making 
process  or  losing  key  clues  that  could  enhance 
efficiency,  prevent  financial  loss  or  -  in  the 
most  significant  case  -  forestall  attack  or  win 
battles . 11 


11  Greiper  and  Sauter,  The  Business  of  Connecting  Dots:  The  $1 
Billion  Intelligence  and  Security  Informatics/Analytics  Market,  6. 


15 


The  complexity  of  these  data  sources  makes  it  difficult 
to  determine  relationships;  for  example,  building  link 
charts  identifying  relationships  from  various  resources, 
handling  massive  quantities  of  information,  identifying 
inconsistencies  between  interviews,  and  extrapolating 
timelines  from  them  are  all  time-consuming  and  cumbersome  to 
accomplish . 

The  20-minute  work-up  is  a  process  in  which  an  officer 
in  the  field  may  have  made  a  traffic  stop  and  needs  an  in- 
depth  workup  on  possible  connections  to  terrorism  or 
narcotics  trafficking  about  an  individual,  their  vehicle, 
and  in  the  case  of  semi  trucks,  the  owner (s)  of  the  cab  and 
trailer.  Legal  restrictions  establish  how  long  such  a  field 
traffic  stop  can  last.  Thus,  extreme  urgency  ensues  in 
accomplishing  these  tasks.  While  not  necessarily  complex  in 
theory,  even  activities  such  as  the  20-minute  work-up  can  be 
complex  with  regard  to  the  types  of  queries  and  the 
multitude  of  data  sources  which  could  be  (and  often  are) 
explored.  Senior  analysts  are  quite  skilled  in  the  proper 
process  and  sequence  of  queries  to  submit  in  order  to 
identify  the  most  significant  amount  of  information  in  the 
least  amount  of  time. 

Current  inefficiencies  related  to  a  20-minute  work-up 
include  the  need  for  multiple  analysts  to  work 
collaboratively  on  a  single  call.  Basically,  they  split  up 
the  effort  due,  in  part,  to  cumbersome  data  access  issues 
with  their  systems  to  collect  as  much  relevant  information 
as  possible  in  the  permitted  timeframe.  Standard  operating 
procedures  (SOPs)  call  for  certain  internal  data  sources  to 
be  queried  on  20-minute  work-ups,  as  well. 
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The  ISP  has  identified  more  than  80  unique  databases 
throughout  the  state  which  are  "owned"  by  ISP  personnel. 
This  represents  a  tremendous  amount  of  information,  yet  most 
analysts  and  users  only  have  access  to  a  handful  of  these 
data  sources.  In  addition,  other  state  agencies  and 
municipal  law  enforcement  entities  have  access  to  their  own 
unique  and  valuable  data  sources,  not  to  mention  the 
previously  noted  and  valued  information  via  the  open  web  or 
paid  subscription  sources. 

The  complexity  of  many  cases  makes  it  difficult  to 
determine  relationships.  The  link  chart  pulls  together 
information  from  various  resources  and  gives  the  analyst  a 
view  of  some  of  these  relationships.  Also,  complexity 
applies  to  the  massive  quantity  of  information  associated 
with  a  particular  case.  A  good  case  in  point  may  be  a 
murder  case  the  Illinois  State  Police  investigated  in 
Streator,  Illinois.  Countless  reports  were  completed  over  a 
period  of  two  years  by  scores  of  investigators.  Thousands 
of  documents  were  produced  including  transcripts  of 
interviews  with  a  variety  of  people  (now  captured  as  PDF 
files)  .  While  extracting  a  list  of  all  the  names  produced 
through  interviews  along  with  the  prevalence  of  each  as  well 
as  their  interrelationship  is  extremely  valuable,  it  is  also 
inherently  labor  intensive.  This  work  is  done  by  the  ISP 
manually,  and  takes  far  too  long.  Identifying 
inconsistencies  between  interviews  and  extrapolating 
timelines  from  them  is  another  analysis  that  is  time- 
consuming  and  cumbersome  to  accomplish  today. 

In  summary,  this  thesis  will  evaluate  if  the  people, 
the  nature  of  their  work,  their  goals,  their  roles,  their 
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organizations,  their  experience  and  finally  their  training; 
in  other  words  the  "context"  of  the  user  needs  to  be  the 
focus  of  solving  the  information  overload  problem  as  opposed 
to  the  traditional  approach  of  focusing  on  the  data  and  how 
it  is  structured  and  shared.  Hallowell  explains  that  the 
personal,  social,  and  organizational  disruptions  need  to  be 
factored  into  any  solution  geared  toward  information 
overload.12  Likewise,  Buck  suggests  that  the  focus  on  the 
prevalence  of  too  much  information  suggests  a  focus  on  the 
data,  concluding  the  data  is  not  the  problem;  the  problem  is 
a  lack  of  time  for  the  consumer 

to  absorb  and  handle  the  data.  Exploiting  a  technology 
solution  to  work  as  the  user  works  will  assist  in 
manipulating  the  volume  of  data  that  exists.13 

D .  RESEARCH  QUESTIONS 

This  thesis  will  study  the  effects  of  Circular 
Reporting  and  Complexity  of  Data  sources  and  evaluate  a 
context-based  method  used  to  mitigate  information  overload. 
Context-based  methodology  is  a  technology  approach  that  may 
solve  the  information  overload  problem  that  is  occurring 
within  state  fusion  centers.  The  information  overload 
problem  is  one  that  will  require  an  understanding,  capturing 
and  leveraging  of  the  knowledge  about  the  user's  context. 

Most  attempts  to  solve  the  information  overload  problem 
have  neglected  the  most  critical  pieces  of  any  solution:  the 

12  Edward  M.  Hallowell,  "Overloaded  Circuits:  Why  Smart  People 
Underperform,"  Harvard  Business  Journal  83,  no.  1  (January,  2005),  54- 
62  . 

13  Brian  Buck,  "Context:  In  Startups,  Software,  &  Sense-Making." 
(Powerpoint,  Distinguished  Entrepreneur  Lecture,  University  of  Illinois 
Urbana-Champaign,  January  19,  2006) . 
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people,  the  nature  of  their  work,  their  goals,  their  roles, 
their  organizations,  their  experience,  and  finally  their 
training.  In  other  words,  the  "context"  of  the  user  has 
been  neglected.  In  a  highly  technical  field  this  is  not 
unusual.  Researchers  and  practitioners  have  been  trained  in 
their  own  technical  field,  not  in  psychology,  sociology, 
ethnography,  and  human  factors.  The  argument  here  is  for  an 
inclusive  and  multi-disciplinary  approach  to  an  inherently 
multi-disciplinary  problem.  Technology  developed  around  a 
context-based  approach  inverts  the  usual  methodology. 
Instead  of  starting  with  the  technology,  and  naively 
believing  all  problems  will  be  solved  if  everyone  could 
retrieve  and  share  information  with  others,  the  context- 
based  approach  starts  with  the  user  and  the  user's  tasks, 
roles,  and  goals.14  The  contextual  approach  then  dictates 
what  type  of  analytics  and  visualizations  need  to  be  brought 
to  bear  to  create  an  effective  solution  to  a  particular 
problem.  The  "bibles"  of  information  visualization  by 
Edward  Tufte  ( The  Visual  Display  of  Quantitative 
Information;  Envisioning  Information;  Visual  Explanations : 
Images  and  Quantities ,  Evidence  and  Narrative ;  and  Beautiful 
Evidence) 15  bring  forth  the  concept  that  information 
visualizations  represent  visual  answers  to  specific 
questions;  the  contextual  approach  is  used  to  determine  what 

14  Keith  Jarvelin,  Peter  Ingerwesen  and  Nick  Belkin,  "Information 
Retrieval  in  Contexts"  (Sheffield,  England,  SIGIR  2004  IRiX  Workshop, 
July  29,  2004 . ) 

http://ir.dcs.gla.ac.uk/ context/ I RinCon text  WorkshopNotes_SIGIR2004 . pdf 
(accessed  March  17,  2007) . 
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are  the  important  questions,  and  these  questions  determine 
which  analytics  and  visualizations  can  serve  to  answer  them. 

Every  instance  of  information  overload  happens  with  a 
specific  person,  enqaqed  in  a  specific  task,  with  specific 
goals,  engaged  with  specific  information  resources.  The 
incredible  variety  of  these  elements  implies  that  there  is 
no  "one-size-f its-all"  solution;  the  most  effective  solution 
will  itself  be  very  specific,  and  tailored  to  the  user, 
task,  role,  goals,  and  sources. 


-*-5  For  Tufte's  major  works  see:  Edward  R.  Tufte,  The  Visual  Display 
of  Quantitative  Information,  2nd  Edition,  (Cheshire  CT:  Graphics  Press, 
2001);  Envisioning  Information,  (Cheshire  CT:  Graphics  Press,  1990); 
Visual  Explanations :  Images  and  Quantities,  Evidence  and  Narrative, 
(Cheshire  CT:  Graphics  Press,  1997);  Beautiful  Evidence,  (Cheshire  CT: 
Graphics  Press,  2006) . 
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II.  LITERATURE  REVIEW 


A.  OVERVIEW 

With  the  explosion  of  available  data  from  the  Internet, 
news  feeds,  internal  data  sources,  and  sensors,  it  has 
become  increasingly  difficult  to  keep  up  with  the  amount  of 
arriving  data,  extract  actionable  information,  and  integrate 
it  with  prior  knowledge  (manually) .  Add  to  that  the 
pressures  of  today' s  fusion  center  climate  —multi-tasking, 
long  hours,  limited  resources,  and  mental  fatigue—  and  it 
becomes  clear  that  analysts,  police  officers  and  executives' 
ability  to  make  rapid,  sound  decisions  is  severely 
compromised  by  information  overload.  As  described  in  great 
detail  within  Chapter  1,  the  Illinois  State  Police  believe 
that  information  overload  is  primarily  created  by  two 
phenomena,  circular  reporting,  and  complexity  of  data 
sources . 

In  the  past  we  have  attempted  to  address  the  problem  of 
information  overload  by  either  adding  analysts  or  by 
warehousing  and  then  mining  the  warehoused  data. 
Unfortunately,  both  approaches  fell  short  of  the  desired 
objective  --  utilization  of  a  human-computer  environment  to 
address  the  problem  of  information  overload.  By  adding  more 
analysts,  the  aim  was  to  process  greater  volumes  of 
information.  In  utilizing  this  approach  we  have  been 
disappointed  to  find  that  some  decision-making  simply  cannot 
be  divided  among  multiple  workers.  In  effect,  all  we  have 
done  is  hire  more  workers  to  review  and  evaluate  the  same 
information,  in  essence  just  spreading  the  problem  around. 
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More  often  than  not,  a  decision  related  to  complex 
circumstances  must  be  made  by  an  individual  that 
incorporates  all  relevant  details.  Asking  multiple  analysts 
to  collaborate  to  produce  a  detailed  model  of  this  kind  is  a 
losing  proposition  since  today' s  analysts  are  already 
spending  the  majority  of  their  time  simply  gathering 
information.  As  a  result,  little  time  remains  for  actual 
analysis  and  sharing  of  information.  Further  reducing  the 
time  available  for  analysis  will  not  improve  the  quality  of 
the  intelligence  gained  from  the  data. 

For  that  reason,  mining  warehoused  data  is  more 
appealing,  though  that  strategy  also  suffers  its  own  set  of 
flaws.  According  to  Kustermann,  unstructured  data  - 
anything  that  resides  outside  a  database,  including  valuable 
data  from  emails,  web  pages,  word  processing  documents,  and 
more,  and  which  are  estimated  to  be  80  percent  of  the 
average  intelligence  data  assets  -  are  integral  to 
operations  but  are  by  definition  outside  the  reach  of 
warehouse-mining  tools.16  Finally,  and  crucially, 
conventional  analytics  operate  in  a  user-driven  mode  on 
large  volumes  of  historical  data  but  cannot  process  real¬ 
time  streams  to  deliver  up-to-the-minute  results. 
Consequently,  fusion  centers  are  trapped  in  reactive,  not 
proactive,  decision-making  environments. 

The  problem  of  information  overload  is  recognized  as  a 
problem  within  many  fields,  and  as  a  result  there  are  many 
categories  of  the  literature  that  address  parts  of  the 
problem.  The  nature  of  the  problem  itself  involves 


16  Aaron  Kustermann  ('Senior  Intelligence  Advisor,  Illinois  State 
Police),  Interview  by  Author,  November  17,  2006. 
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technology,  people,  and  organizations,  and  it  is  not 
surprising  that  different  specialized  fields  have  attempted 
to  tackle  the  portion  of  the  information  overload  problem 
that  lies  within  their  domain. 

B.  PRIMARY  FIELDS 

Below  are  the  primary  fields  that  have  conducted 
research  studying  various  aspects  of  information  overload. 
Although  the  phenomena  of  circular  reporting  and  complexity 
of  data  sources  are  not  new,  there  has  been  little  written 
regarding  their  link  to  information  overload. 

1 .  Information  Science  &  Information  Retrieval 

These  two  fields  both  deal  with  the  collection, 
classification,  retrieval,  and  dissemination  of  information. 
Strengths  of  this  literature  deal  with  making  more  effective 
mechanisms  to  organize  and  retrieve  information.  Weaknesses 
lie  in  three  areas:  first,  incorrectly  assuming  that  having 
perfect  classification  of  storage  and  precision  &  recall  in 
retrieval  solves  the  information  overload  problem  (users  can 
still  be  overwhelmed  with  vast  amounts  of  very  precisely 
relevant  information) ;  second,  in  not  considering  factors 
relating  to  the  information  needs  and  processing 
capabilities  of  users  and  their  organizations;  and  third,  in 
dealing  primarily  with  the  written  word  (unstructured  data 
in  the  form  of  text)  and  not  addressing  issues  regarding 
structured  data.  These  fields  do  not  deal  with  the  "circular 
reporting"  issue  directly;  the  essence  of  circular  reporting 
involves  an  analysis  of  actual  content  to  identify  which 
pieces  of  information  are  duplicative  and  which  are  not,  and 

in  particular  involves  how  additional  related  information 
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and  renditions  of  stories/news  items  change  over  time. 
Neither  of  these  critical  and  nuanced  essentials  is  dealt 
with  in  these  two  fields.  Neither  of  these  two  fields  deals 
with  "complexity  of  data  sources"  as  it  relates  to 
structured  data  sources. 

2 .  Information  Integration 

This  field  deals  with  the  complexities  involved  in  the 
technology  side  of  bringing  together  various  data  sources. 
The  strengths  of  this  literature  deal  with  the  technological 
aspect  of  constructing  solutions  for  the  integration  of 
structured  (database)  information  issues.  Weaknesses 

involve  assuming  that  if  all  systems  have  been  integrated 
that  overload  issues  are  solved  (not  true,  since  additional 
analytics  are  needed  to  avoid  analyst  overload  problems  even 
with  perfectly  integrated  systems) ;  not  dealing  with  text; 
and  not  considering  factors  relating  to  the  information 
needs  and  processing  capabilities  of  users  and  their 
organizations.  This  field  is  not  primarily  concerned  with 
unstructured  data  and  text,  and  thus  does  not  deal  with 
issues  of  "circular  reporting."  This  field  does  address 
parts  of  the  "complexity  of  data"  issue  regarding 
technological  integration,  but  does  not  deal  with  the 
analytical  (e.g.,  entity  analytics  for  resolving  duplicate 
person  records)  or  cognitive  aspects. 

3 .  Information  Architecture  &  the  Semantic  Web 

This  field  sprang  out  of  work  in  the  Human-Computer 
Interaction  (HCI)  and  Usability  fields  which  dealt  with 
issues  regarding  the  organization  of  and  navigation  within 

information  environments.  Strengths  include  the  user 
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information  needs  in  the  design  of  such  environments. 
Weaknesses  include  the  assumption  that  there  will  be  a 
singular  information  environment  and  that  overload  issues 
can  be  addressed  with  proper  organization  and  navigation;  a 
lack  of  exploration  of  integration  issues  involving  multiple 
systems  and  sources  of  data;  and  an  assumption  that  the 
problem  can  be  solved  by  the  content  creator/publisher ,  when 
in  fact  a  key  element  of  the  problem  involves  the 
consumption  of  information  from  a  large  number  of 
information  sources  over  which  there  may  be  no  control.  This 
field  does  not  deal  with  the  specific  of  "circular 
reporting,"  which  involves  the  analysis  and  integration  of 
multiple  sources  of  textual  data  over  time.  It  does  do  some 
work  on  the  issue  of  "complexity  of  data"  as  regards  single 
data  sources  from  cognitive  aspects,  but  does  not  deal  with 
multiple  sources  or  data. 

4.  Human-Computer  Interaction  (HCI) ,  User  Interface 
Design  (UI)  &  Cognitive  Psychology 

These  fields  are  primarily  about  the  effective  design 
of  interfaces  for  information  interaction  that  take  into 
account  user  information  needs  and  user  cognitive  and 
perceptual  abilities.  Strengths  include  the  fundamental 
recognition  that  information  overload  problems  are  as  much 
or  more  about  user  needs  and  capabilities  as  they  are  about 
technology  issues.  Weaknesses  include  the  fact  that 
information  overload  manifests  in  many  different  ways 
depending  on  task,  and  the  literature  is  short  on 
substantive  research  on  information  overload  specifically  in 
intelligence  analysis  tasks;  and  the  fact  that  analytics  are 
not  just  effective  and  efficient  interfaces  for  interaction 
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but  are  necessary  to  solve  information  overload  problems. 
These  fields  teach  cognitive  principles  of  design  that  can 
be  applied  to  solving  "circular  reporting"  and  "complexity 
of  data"  but  the  literature  is  sparse  to  non-existent 
regarding  these  two  specific  tasks/problem  areas. 

5 .  Information  Visualization  &  Visual  Analytics 

These  fields  deal  with  the  effective  and  usable  design 
of  complex  graphical  visualizations  designed  to  afford 
insight  and  leverage  human  visual  pattern  recognition 
capabilities.  Strengths  include  the  ability  to  represent 
complex  and  highly  multidimensional  information  in  a  variety 
of  effective  ways.  Weaknesses  include  the  inability  to 
handle  very  large  scale  visualization  (frequent  in 
intelligence  analysis) ;  an  emphasis  on  a  data-directed 
approach  to  visualization  construction  rather  than  a  focus 
on  user  information  and  decision-making  needs;  and  a 
reliance  on  the  human  visual  system  to  find  patterns  in  a 
visualization  as  opposed  to  employing  analytics  to  find 
patterns  and  using  visualizations  as  visual  explanations. 
The  "circular  reporting"  issue  is  not  addressed  by  this 
literature.  The  "complexity  of  data"  is  addressed  to  some 
degree,  but  primarily  with  respect  to  visual  representations 
of  complex  data  and  not  regarding  analytical  or  integration 
aspects . 

6.  Intelligence  Analysis 

This  field  deals  with  the  study  of  policy, 
organizational,  and  analyst  issues  regarding  the  effective 
intelligence  analysis  processing  cycle.  Strengths  include 

some  focus  on  specific  tasks  faced  by  analysts,  although 
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there  is  some  controversy  within  the  literature  regarding 
whether  information  overload  is  actually  a  problem. 
Weaknesses  include  a  lack  of  emphasis  on  cognitive  and 
perceptual  aspects  of  information  overload.  One  aspect  of 
the  "circular  reporting"  issue  is  described  (in  the  context 
of  HUMINT  source  reporting  provenance)  but  not  the  larger 
issues  involving  public  media  and  agency  dissemination.  The 
"complexity  of  data"  issue  is  dealt  with  partially  via 
descriptions  and  more  from  an  organizational  rather  than 
individual  analyst  perspective,  but  not  from  an  analytic, 
integrative  or  cognitive  perspective. 

7 .  Organization  Science 

This  field  deals  with  the  study  of  effective  decision¬ 
making  within  organizations.  Strengths  include  the 
recognition  of  and  original  work  on  information  overload  as 
a  problem  situated  within  the  context  of  organizational 
structures,  business  processes,  and  tasks;  the  description 
and  study  of  the  effects  of  the  problem  within 
organizations;  and  the  exploration  of  remediation  steps  that 
can  be  taken  by  organizations  and  individuals  to  mitigate 
such  effects.  Weaknesses  include  lack  of  literature  dealing 
with  increasing  the  ability  of  individuals  and  organizations 
to  operate  effectively  in  environments  with  a  high  degree  of 
information  flow,  as  opposed  to  finding  ways  to  reduce  the 
information  flow  to  accommodate  current  limits.  The 
"circular  reporting"  issue  is  absent  from  this  field' s 
research,  and  the  "complexity  of  data"  issue  is  partially 
described  but  technological  and  analytic  research  is  not 
present . 
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8 .  Augmented  Cognition 


This  is  an  emerging  field  (most  work  within  the  last 
seven  years  or  so)  with  a  small  literature  base,  founded  on 
the  proposition  that  much  of  the  information  overload 
problem  source  comes  from  a  cognitive  overload  and  can  be 
addressed  by  detecting  (physiologically)  a  user' s  current 
cognitive  processing  state  and  by  dynamically  and 
selectively  altering  or  reducing  the  information  flow  in 
response.  Strengths  include  the  inherently  multi¬ 
disciplinary  approach,  but  the  literature  is  still  small 
compared  to  the  other  fields.  Weaknesses  include  an 
emphasis  on  the  forms  of  information  overload  that  relate  to 
cognitive  overload,  when  there  are  many  other  manifestations 
of  information  overload.  This  field  does  not  deal  with 
"circular  reporting"  at  all.  This  field  does  deal  with  the 
human  cognitive  aspects  of  "complexity  of  data"  but 
exclusively  with  those  aspects,  not  touching  explicitly  upon 
the  integration  and  analytic  aspects. 

C.  LITERATURE  REVIEW  SUMMARY 

An  important  characteristic  about  the  literature 
regarding  the  problem  of  "information  overload"  is  that 
there  is  not  just  one  thread,  but  rather  multiple  views  of 
the  problem,  each  from  the  perspective  of  a  different  field, 
each  with  its  own  specific  concerns.  Unfortunately, 
however,  the  nature  of  the  information  overload  problem 
itself  is  inherently  multi-disciplinary;  it  spans  many 
disciplines  because  it  involves  technology,  people, 
organizations,  and  their  missions,  roles,  and  tasks.  Thus, 
almost  all  of  the  literature  is  narrow  and  omits  discussion 
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of  many  aspects  of  the  problem.  Unexplored  areas  primarily 
relate  to  research  involving  combinations  of  these 
disciplines . 

Information  overload  manifests  in  many  different  ways 
that  are  particular  to  an  organization's  mission,  role,  and 
tasks  within  which  individual's  work.  The  different 
categories  of  literature  above  have  not  dealt  with  the 
particulars  of  the  "circular  reporting"  and  the  "complexity 
of  data  sources"  problems.  Different  fields  deal  with  it  in 
their  literature,  each  from  their  own  particular  field's 
perspective,  but  none  addresses  it  holistically.  One  is 
reminded  of  the  parable  of  "The  Blind  Men  and  the 
Elephant,"17  as  while  there  is  a  line  of  research  within 
each  of  these  disciplines  somehow  a  collection  of  each 
discipline-specific  view  does  not  yield  an  overall  one  that 
comprehends  the  elephantine  information  overload  problem. 


Figure  4.  The  Blind  Men  and  the  Elephant18 
(reprinted  with  permission) 


17  For  a  reminder  of  the  impact  of  the  parable,  see  the  poem  written 
by  John  Saxte  found  at 

http : // www . cs . rice . edu/~ssiyer /minstrels /poems/ 1179. html  (accessed 
January  7,  2008) . 

18  "The  Blind  Men  and  the  Elephant, " 

http : // www . j  ainworld . com/ education/ j  uni or s/ j  unll 9il .gif  (accessed 
November  7,  2007) . 
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Next,  this  writing  will  define  the  components  of  the 
"information  overload"  problem  in  order  to  provide  a  context 
for  examining  the  relationships  within  the  various  pieces  of 
literature;  it  will  explore  the  affinity  between  each 
component  and  a  particular  discipline;  and  then  it  will 
observe  and  comment  on  the  various  writings  from  each  of 
these  disciplines.  Finally,  those  few  more  recent  multi¬ 
disciplinary  writings  will  be  noted  that  come  closer  to 
addressing  the  problem  as  a  whole. 

1.  Information  Collection,  Organization 

By  "some  estimates  it  takes  the  world  only  15  minutes 
to  create  a  quantity  of  information  equivalent  to  that 
currently  held  in  the  Library  of  Congress."19  There  is  no 
question  that  the  pace  of  information  growth  is 
accelerating,  to  the  point  where  the  perception  of  the 
information  overload  problem  is  pervasive.  It  is 

traditional  to  think  of  the  overload  as  consisting  of 
structured  information  in  databases  and  data  warehouses,  but 
the  largest  constituent,  and  fastest  growing,  is 
unstructured  information  (text) : 

Colin  White  related  the  following  facts:20 

•  80  percent  of  business  is  conducted  on 
unstructured  information  (Gartner  Group) . 

•  85  percent  of  all  data  stored  is  held  in  an 
unstructured  format  (Butler  Group) . 


19  Jonathan  Spira,  "Lost  Knowledge:  File  Not  found,"  KMWorld  (2006)", 
accessed  February  2,  2007)  . 

20  Colin  White,  "Consolidating,  Accessing  and  Analyzing  Unstructured 
Data,"  Business  Intelligence  b-Eye  Newsletter  (December  12,  2005), 
http://www.b-eye-network.com/view/2098  (accessed  February  1,  2007). 
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•  Unstructured  data  doubles  every  three  months 
(Gartner  Group) . 

•  7  million  web  pages  are  added  every  day 
(Gartner  Group) 

Many  observers  think  that  enormous  growth  in 
unstructured  information  will  itself  be  dwarfed  by  the 
coming  growth  in  real-time  streams  of  sensor  information.21 
Refer  to  Harry  Goldstein's  commentary  where  he  describes  the 
vision/prediction  of  a  "sensor  planet  of  2020  teems  with 
billions  of  wireless  ultra  wideband  communications  nodes 
connected  to  countless  pinhead  size  cameras,  microphones, 
motion  detectors,  and  biometric  and  other  sensors  to  form  a 
fine-grained  mesh  of  networks  that  cover  every  square 
millimeter  of  the  globe."22 

Workers  in  Information  Science  and  Information 
Retrieval  are  addressing  this  component  of  the  information 
overload  problem. 

2 .  Information  Science 

Information  science  is  "an  interdisciplinary  science 
primarily  concerned  with  the  collection,  classification, 
manipulation,  storage,  retrieval  and  dissemination  of 


21  Vernor  Vinge,  "Synthetic  Serendipity,"  IEEE  Spectrum  41,  no.  7 
(July  2004),  35-44.  http://ieeexplore.ieee.org/  (accessed  January  31, 
2007)  . 

22  Harry  Goldstein,  "Mike  Villa's  World,"  IEEE  Spectrum  41,  no.  7 
(July  2004):  45.  http://ieeexplore.ieee.org/  (accessed  January  31, 
2007)  . 
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information."23  The  literature  in  this  field  includes  works 
from  several  decades  ago  up  through  the  current  day.  There 
are  some  who  controversially  believe  that  the  problem  is 
simply  one  of  insufficient  training.  Bundy  believes,  that 
"people  often  lack  the  understandings  and  skills  to 
identify,  locate,  access,  evaluate,  and  then  apply  the 
needed  information."24  This  is  an  extreme  position  along 
the  continuum  of  literature  in  Information  Science. 

Most  of  the  works  in  this  field  deal  with  more 
effective  methods  in  indexing,  metadata  creation,  and 
search . 

3 .  Information  Retrieval 

Information  retrieval  is  itself  an  interdisciplinary 
field  that  is  primarily  concerned  with  the  science  of 
searching  for  documents  and  information  within  them,  and 
incorporates  aspects  of  "cognitive  psychology,  information 
architecture,  information  design,  human  information 
behavior,  linguistics,  information  science,  computer 
science,  librarianship  and  statistics."25 


23  Ben  Campbell,  "Information  Science,"  Information  Architecture 
Glossary , 

http : // iml . j  ou . uf 1 . edu/pro j  ects/FallO 6 /Campbell/ resources /glossary . html 
(accessed  February  1,  2007) . 

24  Alan  Bundy,  "Drowning  in  Information,  Starved  for  Knowledge: 
Information  Literacy,  Not  Technology,  is  the  Issue"  (Melbourne, 
Australia,  10th  VALA  Biennial  Conference  and  Exhibition,  February  16-18, 
2000 )  http : / /www . vala . org . au/vala2000/ 2000pdf /Bundy . PDF  (accessed 
January  24,  2007) . 

25  Zdenka  Linkova,  "European  Summer  School  in  Information  Retrieval 
ESSIR  2005,"  Technical  Report  no.  949,  (Institute  of  Computer  Science, 
Academy  of  Sciences  of  the  Czech  Republic,  2006) 

http : / /www .cs. cas.cz/cgi- 

bin/ extf tp?f tp ://ftp.cs.cas. cz/pub/ reports /v94  9-06 . pdf  (accessed 
February  7,  2007) . 
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This  area  has  extensive  literature,  much  of  which 
focuses  on  algorithms  for  improved  precision  and  recall  in 
information  retrieval.  The  literature  dates  back  into  the 
middle  of  the  last  century  wherein  Gerard  B.  Salton' s 
writings26  form  the  basis  for  much  work  that  followed,  as 
well  as  much  active  work  today.  The  National  Institute  of 
Standards  sponsored  a  series  of  conferences  and  competitions 
(Text  Retrieval  Conference,  or  TREC,  described  at 
http://trec.nist.gov/)  for  many  years.  There  are  many  other 
academic  publications  and  conferences  focused  on  various 
aspects  of  information  retrieval. 

There  is  not  much  controversy  within  this  sub¬ 
literature,  but  there  is  a  significant  amount  of  ongoing 
innovation,  much  of  it  spurred  by  the  growth  of  the  Web. 
The  most  significant  unexplored  areas  represent  ones  that 
typically  are  outside  the  primary  interests  of  this 
discipline,  which  is  focused  on  only  one  small  part  of  the 
information  overload  problem,  retrieval  effectiveness. 

4 .  Information  Integration 

The  task  of  information  integration  involves  bringing 
together  various  data  sources  (mostly  structured  databases) 
with  a  goal  of  enabling  information  fusion.  Many  of  the 
detailed  activities  involve  database  schema  mapping, 
architectures  for  data  integration,  and  more  recently,  the 
use  of  metadata  and  semantic  technology  to  achieve  a  goal  of 
semantic  interoperability. 


26  For  more  on  Salton' s  work  see:  Gerard  Salton,  "Export  Systems  and 
Information  Retrieval,"  ACM  SIGIR  Forum  21,  no.  1  (March  1987) :  3-9. 
http : //portal . acm. org/citation . cfm?id=30076  (accessed  January  7,  2008). 
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This  literature,  although  active  and  with  roots  going 
back  to  the  1960s,  is  rather  narrow  from  the  perspective  of 
the  information  overload  problem.  An  example  is  Information 
Integration  for  Counter  Terrorism  Activities :  The 
Requirement  for  Context  Mediation.  The  focus  of  this  type 
of  work  is  on  integration  from  a  contextually  driven 
perspective  of  intelligence  analysts'’  needs,  but  doesn't 
explicitly  address  cause,  issues,  or  remediation  of 
information  overload  problems.27  Madnick' s  work  on  the 
Context  INterchange  (COIN)  system  and  the  concept  of  a 
Context  Mediator  is  the  most  significant  work28  in  this 
field  that  moves  outside  the  realm  of  purely  technical  and 
algorithmic  concerns.  The  importance  of  beginning  to 
include  the  user  and  the  user's  concerns,  tasks,  goals,  and 
perceptual  and  cognitive  capabilities  bodes  well  for 
continued  work  in  this  field. 

5 .  Information  Architecture 

This  is  a  relatively  young  field,  born  in  the  1990s 
around  the  birth  and  growth  of  the  Web,  and  beginning  to 
blossom  in  the  last  five  years.  The  literature  is  primarily 
about  methods  for  design  of  information  environments,  using 
articulated  design  principles  and  leveraging  metadata. 


27  Nazli  Choucri  and  others,  "Information  Integration  for  Counter 
Terrorism  Activities:  The  Requirement  for  Context  Mediation,"  MIT  Sloan 
Working  Paper  no.  4479-03,  (Cambridge,  MA:  MIT  Sloan  School  of 
Management,  2003) . 

http : / /papers . ssrn . com/ sol 3 /papers . cfm?abstract%5Fid=52  97  03  (accessed 
February  1,  2007) . 

28  Stuart  E.  Madnick  and  Lupu  Mihai,  "Using  Semantic  Web  Tools  for 
Context  Interchange,"  MIT  Sloan  Research  Paper  no.  4668-07,  (Cambridge, 
MA:  Massachusetts  Institute  of  Technology,  September  2007) . 

http : / /papers . ssrn . com/ sol 3 /papers . cfm?abstract_id=1075245  (accessed 
January  7,  2008) 
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taxonomies  and  ontologies  to  create  environments  that  can 
meet  specific  user  needs.29 

There  is  not  much  controversy  in  this  field,  but  as  a 
young  research  area  it  is  still  finding  its  own  voice  among 
the  many  related  fields.  Much  of  the  writing  relates  to 
technology  issues,  or  strongly  overlaps  work  in  the  HCI  and 
Usability  fields.  The  extent  to  which  this  body  of  work 
deals  with  information  overload  is  in  the  almost  implicit 
assumption  that  having  "more  organized  information 
environments"  will  be  a,  if  not  the,  major  factor  in 
alleviating  the  problem.  Thus  there  is  little  exploration 
on  the  nature  of  the  information  overload  problem  per  se. 

This  author  would  characterize  the  Semantic  Web  work 
espoused  by  Tim  Berners-Lee  as  dealing  with  a  large-scale 
and  Web-specific  version  of  Information  Architecture 
issues . 30 

6.  Human-Computer  Interaction  (HCI),  User  Interface 
Design  (UI) ,  Cognitive  Psychology  and  Perceptual 
Capabilities 

These  two  components  are  combined,  although  UI  Design 
focuses  more  on  the  production  of  artifacts  for  user 
interfaces  and  devices,  and  the  latter  component  represents 
consideration  of  fundamental  human  capabilities,  because  the 
fields  that  deal  primarily  with  these  components  tend  to 
focus  to  a  great  degree  on  combinations  of  these.  HCI  and 
Cognitive  Psychology  literature  both  deal  extensively  with 

29  Alan  Gilchrist  and  Barry  Mahon,  eds .  ,  Information  Architecture: 
Designing  Information  Environments  for  Purpose  (London:  Facet 
Publishing,  2004) . 

33  Tim  Berners-Lee,  "Semantic  Web  Road  Map." 
http://www.w3.org/DesignIssues/Semantic.html  (accessed  March  17,  2007). 
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these  issues,  although  HCI  tends  to  also  deal  with  the 
design  and  evaluation  of  user  interfaces  and  devices. 

Drucker,  in  a  position  paper  for  a  conference  workshop, 
does  a  good  job  of  categorizing  the  issues.  Most 
importantly,  he  provides  a  description  of  some  fundamental 
information  tasks  that  form  the  milieu  in  which  information 
overload  problems  manifest  themselves.31  These  tasks, 
although  not  domain  specific  in  the  sense  of  being  concerned 
with  intelligence  analysts  involved  in  counter-terrorism 
activities,  nonetheless  are  an  important  move  to  understand 
the  complexity  of  information  overload  via  a  "divide  and 
conquer"  approach  that  reflects  the  intrinsically  varied  and 
complex  nature  of  the  problem. 

HCI  literature  goes  back  decades  and  had  its  roots  in 
human  factors  work  involving  such  things  as  design  of 
aviation  displays,  etc. 

7 .  Information  Visualization 

The  literature  of  Information  Visualization  draws 
heavily  from  its  heritage  in  the  use  of  computer  graphics  to 
display  abstract  visualizations  of  complex  information,  with 
a  belief  that  the  human  visual  pattern  processing 
capabilities  will  be  the  key  ingredient  to  help  humans  find 
structure  in  masses  of  complex  data. 


31  Steven  M.  Drucker,  "Coping  with  Information  Overload  in  the  New 
Interface  Era,"  (Montreal,  Canada,  CHI2006,  What  is  the  Next  Generation 
of  Human  Computer  Interaction?  April  23,  2006) . 

http : / /www . cs . tufts . edu/~ j  acob/ workshop /papers /drucker . pdf  (accessed 
January  31,  2007) . 
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Much  of  the  computer  related  literature  found  its 
genesis  in  the  work  of  Stuart  Card  and  others  at  Xerox  PARC 
going  back  to  1960s  and  1970s,  and  continuing  through  the 
present  day.  As  previously  stated,  the  preeminent  voice  in 
this  field  is  Edward  R.  Tufte,  whose  four  books  are 
considered  classics  in  this  field. 

There  are  two  schools  of  thought  in  this  literature, 
which  has  engendered  some  controversy;  one  school  focuses  on 
the  construction  of  information  visualizations  and  tends  to 
pay  little  attention  to  actual  evaluation  of  the 
effectiveness  of  such  visualizations;  the  other  school  not 
only  is  concerned  with  leveraging  the  knowledge  from  the 
literature  on  human  cognitive  and  perceptual  capabilities 
but  also  does  concern  itself  with  achieving  more  results  and 
the  production  of  pretty  pictures. 

Where  there  is  a  weakness  in  this  field  it  lies  in  the 
school  that  gives  little  attention  to  evaluation.  To  some 
extent  the  newly  christened  subfield  of  Visual  Analytics  (in 
part,  a  euphemism  coined  in  a  post-TIA  (Total  Information 
Awareness)  world  for  data  mining,  and  in  part  characterized 
by  a  belief  that  visualization,  more  so  than  underlying 
analytics,  can  be  a  key  to  human  understanding)  has  begun  to 
spawn  a  literature  focused  on  domain-specific  tasks  such  as 
those  facing  intelligence  analysts. 

8 .  Tasks ,  Goals ,  and  Workflows 

A  focus  on  the  actual  tasks,  goals,  and  workflows  of 
practicing  intelligence  analysts  has  been  at  the  center  of  a 
body  of  literature  growing  for  the  last  decade  or  so.  There 
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is  some  controversy,  however,  such  as  intelligence  expert, 
Richard  Heuer' s  assertion. 

On  the  contrary,  my  own  30-plus  years  in  the 
business  of  intelligence  analysis  biased  me  in 
favor  of  the  view  that,  endless  warnings  of 
information  overload  notwithstanding,  there  is  no 
such  thing  as  too  much  information  or  expertise.32 

Despite  the  fact  that  Heuer' s  work  is  one  of  the  "bibles"  of 
intelligence  analysis;  the  rest  of  the  literature  recognizes 
there  are  many  manifestations  of  information  overload  within 
the  tasks,  goals,  and  workflows  of  intelligence  analysts. 

Some  in  this  field  only  represents  a  situational 
positioning  of  what  would  otherwise  be  considered  general 
literature  from  one  of  the  other  fields  mentioned,  without  a 
deep  exploration  in  the  contextual  nature  of  an  analyst's 
job.  Others,  however,  like  Sanquist,  et  al .  and  Patterson, 
et  al .  have  begun  to  take  an  intensely  contextual  look  at 
the  nature  of  and  variety  of  problems  facing  this  type  of 
knowledge  worker.  33  34  In  the  view  of  this  author,  this 
approach,  augmented  with  the  work,  research  results,  and 
techniques  from  the  other  sub-literatures  discussed  herein 


32  Heuer,  Richards  J.,  Jr.,  Psychology  of  Intelligence  Analysis,  1st 
ed.  (Washington,  DC:  Central  Intelligence  Agency,  Center  for  the  Study 
of  Intelligence,  1999),  x-xi .  https://www.cia.gov/library/center-for- 
the-study-of- intelligence/ cs i -publ icat ions /books -and- 

monographs /psycho logy- of -intelligence-analysis /Psychof IntelNew . pdf 
(accessed  January  3,  2008) . 

33  Thomas  F.  Sanquist  and  others,  "Cognitive  Tasks  in  Intelligence 
Analysis:  Use  of  Event  Dwell  Time  to  Characterize  Component  Activities" 
(New  Orleans,  LA,  Human  Factors  and  Ergonomics  Society  48th  Annual 
Meeting,  September  20-24,  2004)  . 

34  Emily  S.  Patterson,  David  D.  Woods  and  David  Tinapple,  "Using 
Cognitive  Task  Analysis  (CTA)  to  Seed  Design  Concepts  for  Intelligence 
Analysts  Under  Data  Overload"  (Santa  Monica,  CA,  Human  Factors  and 
Ergonomics  Society  45th  Annual  Meeting,  October  8-12,  2001) . 
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represent  the  best  hope  of  moving  towards  a  broad,  pan- 
disciplinary  approach  to  making  significant  traction  with 
information  overload  problems. 


9.  Personal,  Social,  and  Organizational  Impacts 

Eppler  and  Mengis  introduce  their  own  literature  survey 
on  information  overload  research  by  explaining  the  concept 
of  information  overload  and  its  many  related  terms: 

In  ordinary  language,  the  term  'information 
overload'  is  often  used  to  convey  the  simple 
notion  of  receiving  too  much  information.  Within 
the  research  community  this  every  day  use  of  the 
term  has  led  to  various  constructs,  synonyms  and 
related  terms  as  for  example  cognitive  overload, 
sensory  overload,  communication  overload, 
knowledge  overload,  or  information  fatigue 
syndrome . 35 

One  of  the  ways  that  researchers  look  at  information 
overload  is  from  a  perspective  of  organizational  impacts 
which  involve  exploration  of  various  communication 
processes.  Eppler  and  Mengis  enumerate  the  representative 
works  in  this  area  as: 


Topic 

Reference 

Meetings 

Allen  G.  Schick,  Lawrence  A. 
Gordon,  and  Susan  Haka, 

" Information  overload :  A  temporal 
approach,"  Accounting  Organizations 
and  Society ,  15:  199-220,  1990. 

33  Martin  J.  Eppler  and  Jeanne  Mengis,  A  Framework  for  Information 
Overload  Research  in  Organizations :  Insights  from  Organization  Science, 
Accounting ,  Marketing ,  MIS,  and  Related  Disciplines  (Lugano, 
Switzerland:  Universita  della  Svizzera  Italiana,  September  2003),  7. 
http : / /www . bul . uni si . ch/ cerca/bul/pubblicazioni/ com/ pdf /wpca0301 . pdf 
(accessed  April  20,  2007) . 
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Telephone 

conversations 


Allen  G.  Schick,  Lawrence  A. 
Gordon,  and  Susan  Haka, 
" Information  overload :  A  temporal 
approach,"  Accounting  Organizations 
and  Society,  15:  199-220,  1990. 


The  use  of 

Groupware 

applications 


Bulletin 
Board  Systems 


Face  to  face 
discussions 


Telephone 

company 

services 


Electronic 

meetings 


Idea 

organization 


Ulrike  Schultz  and  Betty 
Vandenbosch,  " Information  Overload 
in  a  Groupware  Environment:  Now  You 
See  It,  Now  You  Don't,"  Journal  of 
Organizational  Computing  and 
Electronic  Commerce,  1998,  Vol.  8, 
No.  2,  Pages  127-148. 

Starr  Roxanne  Hiltz  and  Murray 
Turoff,  "Structuring  Computer- 
Mediated  Communication  Systems  To 
Avoid  Information  Overload, " 
Communications  of  the  ACM,  Volume 
28,  Issue  7  (July  1985)  Pages:  680 

-  689. _ 

Paul  R.  Sparrow,  " Strategy  and 
cognition:  Understanding  the  role 
of  management  knowledge  structures, 
organizational  memory  and 
information  overload,"  Creativity 
and  Innovation  Management,  8:  140- 

149,  1999 _ 

Rodger  W.  Griffeth,  Kerry  D. 
Carson,  and  Daniel  B.  Marin, 

" Information  overload :  A  test  of  an 
inverted  U  hypothesis  with  hourly 
and  salaried  employees,"  Academy  of 


Kerry  D. 
B.  Marin, 


Management 

1988  . _ 

Mary-Liz 

Gallupe, 


Proceedings , 


Academy  of 
232-237 , 


Mary-Liz  Grise,  and  R.  Brent 
Gallupe,  " Information  overload : 
Addressing  the  productivity  paradox 
in  face-to-face  electronic 
meetings,"  Journal  of  Management 
Information  Systems ,  16  (3)  :  157- 

185,  1999/2000. _ 

Mary-Liz  Grise,  and  R.  Brent 
Gallupe,  " Information  overload : 
Addressing  the  productivity  paradox 
in  face-to-face  electronic 


R.  Brent 
overload : 
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Topic 


Reference 


meetings,"  Journal  of  Management 
Information  Systems ,  16  (3)  :  157- 

185,  1999/2000. 

E-mail 

David  Bawden,  "Information 
overload,"  Library  &  Information 
Briefings,  92. 

C.  Speier,  J.S.  Valacich  and  I. 
Vessey,  " The  influence  of  task 
interruption  on  individual  decision 
making:  An  information  overload 
perspective,"  Decision  Sciences, 
30:  337-359,  1999. 

Peter  J.  Denning,  "Electronic 
junk,"  Communications  of  the  ACM, 
25:  163-168,  1982. 

Management 

consulting 

Morten  T.  Hansen  and  Martine  R. 
Haas,  " Competing  for  Attention  in 
Knowledge  Markets :  Electronic 
Document  Dissemination  in  a 
Management  Consulting  Company, " 
Administrative  Science  Quarterly , 
Nr.  46:  1-28,  2001 

City 

interactions 

Stanley  Milgram,  "The  Experience  of 
Living  in  the  Cities,"  in  Science, 
March,  13:  1461-1468,  1970 

Disclosure 
law,  contract 
complexity, 
legal 

disclaimers 

David  M.  Grether,  Alan  Schwartz, 
and  Louis  L.  Wilde,  " The 
irrelevance  of  information 
overload :  An  analysis  of  search  and 
disclosure,"  Southern  California 
Management  Review,  59:277-303,  1986 

Table  1 .  Eppler  and  Mengis  Literature  Review 


Researchers  have  found  the  performance  of  individuals 
in  decision-making  improves  with  increasing  amount  of 
information,  up  to  a  point,  after  which  performance  actually 
suffers.  Beyond  this  point,  at  the  information  overload 
level,  additional  information  is  not  incorporated,  and  it  is 
difficult  for  the  individual  to  prioritize  and  even  to  deal 
with  and  recall  information  encountered  earlier. 
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Eppler  and  Mengis  and  their  cited  references  provide 
additional  exploration  of  this  topic.  Eppler  and  Mengis 
describe  a  significant  factor  missing  in  the  literature, 
which  is  emphasized  in  this  paper: 

As  contextual  factors  (such  as  industry 
characteristics,  the  firm's  development  stage  or 
the  staff  structure  etc.)  are  of  crucial 
importance  for  the  occurrence  of  overload, 
research  methods  should  be  applied  that  can 
capture  many  of  these  contextual  factors  and  thus 
highlight  interdependencies.  Research  approaches 
that  provide  such  a  'deep  context'  are  missing, 
as  most  literature  on  the  topic  is  experimental, 
survey-based,  or  purely  conceptual.36 

The  importance  of  context,  although  recognized  by 
Eppler  and  Mengis,  has  not  been  significantly  explored  in 
the  research. 

D .  INFORMATION  OVERLOAD 

Table  2  (below)  identifies  each  of  the  (at  least 
partially)  relevant  research  threads  from  various  fields 
that  touch  upon  the  information  overload  problem.  The  Table 
identifies  the  areas  of  investigation  of  each  research 
thread.  It  is  apparent  that  each  research  thread  approaches 
the  problem  from  its  own  unique  perspective,  and  while  there 
is  certainly  some  overlap  between  some  of  the  research 
threads,  no  single  (or  even  pair)  of  research  threads  covers 
the  entire  breadth  and  depth  of  this  problem. 


36  Eppler  and  Mengis,  A  Framework  for  Information  Overload  Research 
in  Organizations :  Insights  from  Organization  Science,  Accounting, 
Marketing,  MIS,  and  Related  Disciplines,  13. 
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Research 

Thread 

Discipline  / 
Field 

Information 

Overload 
(Fusion  Center) 

Information 

Collection, 

Organization 

Information 

Science ; 

Information 

Retrieval 

Organization  of 
collections ; 
indexing  and 
metadata;  search 
and  relevance. 

Information 

Integration 

Computer  Science 

Database 

integration,  schema 
mapping,  and 
semantic 

interoperability . 

Information 

Architecture 

Information 

Architecture 

Structural  design 
of  shared 
information 
environments ; 
usability  and  find 
ability;  focus 
design  of 
information 
artifacts . 

HCI,  UI  Design 

Human-Computer 
Interaction  & 
Usability 

Design  and 
implementation  of 
interfaces  between 

humans  and 
computers;  focus  on 
meeting  user' s 
needs . 

Information 

Visualization 

Information 

Visualization 

Design  and 
presentation  of 
images  and  abstract 
visualizations  to 
enhance  human 
comprehension . 

Human  cognitive  and 

perceptual 

capabilities 

Human  Factors  & 
Cognitive 

Psychology 

Focus  on  the  human 

side  of  interaction 
with  systems; 
understanding 
fundamental  limits 
and  capabilities  of 
humans  for 
perception  and 
cognition;  human 
problem  solving, 
memory,  language. 

43 


Research 

Thread 

Discipline  / 
Field 

Information 

Overload 
(Fusion  Center) 

Personal,  Social 

Psychology  & 

Study  of  individual 

and  Organizational 
Impacts 

Sociology 

and  group 
interactions . 

Tasks,  Goals, 

Intelligence 

Focus  on  problems 

Workflows 

Analysis ; 

Usability  & 
Contextual  Design 

facing  intelligence 
analysts , 

particularly  task- 
specific  analyses. 

Table  2 .  Components  of  Information  Overload  &  Related 


Disciplines 

1 .  Multidisciplinary  Approaches 

There  are  few  multidisciplinary  approaches  being  taken 
in  the  current  research  threads  on  this  topic.  The  one  that 
has  the  broadest  multidisciplinary  approach  is  the  emerging 
field  of  Augmented  Cognition.  Although  it  does  not  yet 
encompass  all  the  other  research  threads,  one  could  make  the 
case  that  because  of  its  intrinsic  focus  on  the  user  and  the 
user's  cognitive  processes;  it  stands  the  best  chance  of  in 
the  future  of  incorporating  the  other  social, 
organizational,  contextual  and  technological  aspects  of  this 
problem . 

2 .  Augmented  Cognition 

Augmented  Cognition  is  a  very  new  field  attempting  a 
fusion  among  several  of  the  disciplines,  specifically  HCI, 
UI  Design,  and  human  cognitive  and  perceptual  capabilities, 
but  still  falls  short  of  exploiting  all  of  the  dimensions  of 
context . 
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Augmented  Cognition  is  a  field  that  does  not 
reside  in  just  one  scientific  discipline  —  it 
draws  from  areas  such  as  neuroscience, 
biopsychology,  cognitive  psychology,  human 
factors,  information  technology,  and  computer 
science.  Each  of  these  fields  has  itself 
undergone  a  substantial  revolution  over  the  past 
forty  years  that  has  allowed  the 
problems/challenges  raised  by  the  researchers  in 
this  field  to  begin  to  be  satisfactorily 
investigated.  Although  there  are  many  individual 
research  projects  that  contributed  to  the  general 
development  and  direction  of  this  field,  several 
multimillion  dollar  efforts  helped  shape  the 
foundation  on  which  the  current  augmented 
cognition  field  is  built.37 


E .  THE  FUTURE 

As  previously  stated  "circular  reporting"  and 
"complexity  of  data  sources"  are  not  new,  however,  little 
has  been  written  regarding  their  link  to  specific 
information  technology  disciplines.  The  absence  of  research 
on  "circular  reporting"  and  "complexity  of  data  sources"  may 
be  due  to  the  fact  that  circular  reporting"  and  "complexity 
of  data  sources"  are  more  prevalent  in  the  post  9/11 
intelligence  community  with  the  emergence  of  fusion  centers 
that  have  created  a  new  set  of  information  overload 
problems.  Research  needs  to  be  conducted  exploring  the 
impact  of  "circular  reporting"  and  "complexity  of  data 
sources"  on  information  overload  and  how  to  best  mitigate 
its  effects. 


37  "History:  Emergence  of  Augmented  Cognition,"  Augmented  Cognition 
International  Society,  http:// www . augmentedcognition . org/ hi story . htm 
(accessed  January  30,  2007) . 
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Ill .  METHODOLOGY 


A.  OVERVIEW 

The  research  portion  of  this  thesis  will  utilize  a 
survey  to  explore  the  effectiveness  of  reducing  information 
overload  in  terms  of  circular  reporting  and  complexity  of 
data  sources  through  the  development  of  technology  using  a 
context-based  approach. 

The  survey  will  measure  the  respondents'  beliefs  and 
opinions  about  the  use  of  a  technology  artifact  built  using 
a  context-based  approach,  in  particular,  the  respondents' 
perceived  value  in  the  implementation  of  the  technology 
artifact  on  the  following  issues:  information  overload, 
circular  reporting,  and  complexity  of  data  sources.  In 
addition,  the  survey  will  measure  the  respondents'  views  of 
the  system' s  usefulness  and  the  ease  with  which  it  can  be 
used,  both  of  which  are  predictors  of  future  technology 
adoption . 

The  survey  will  follow  a  deductive  approach  by 
beginning  with  applied  research  problems  and  ending  with 
empirical  measurements  and  data  analyses  both  of  which  will 
be  expressed  in  Chapter  IV;  the  findings  will  be  discussed 
in  Chapter  V;  and  the  thesis  will  conclude  with  a 
recommendations  section  outlined  in  Chapter  VI. 

B.  POPULATION 

The  sample  population  will  be  made  up  of  34  criminal 
intelligence  analysts.  All  of  the  subjects  are  assigned  to 
the  Illinois  Statewide  Terrorism  and  Intelligence  Center, 
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(STIC) .  The  analysts  are  permanent  employees  of  the 
Illinois  State  Police,  Illinois  National  Guard,  Federal 
Bureau  of  Investigation,  Drug  Enforcement  Administration, 
and  the  Department  of  Homeland  Security.  The  respondents 
have  agreed  to  answer  the  statements  and  understand  they  may 
refuse  to  participate  or  to  answer  any  particular  statement. 

C.  RESEARCH  ASSISTANTS 

It  is  relevant  to  note  that  the  author  has  been 
involved  with  the  conceptual  development  of  the  technology 
artifact  to  be  tested  and  is  a  Sr.  Command  official  within 
the  Illinois  State  Police.  In  this  capacity  the  author  has 
ultimate  responsibility  for  the  oversight  of  the  Statewide 
Terrorism  and  Intelligence  Center  and  its  employees.  In  an 
effort  to  eliminate  perceived  interview  bias  and  prestige 
bias,  the  author  will  be  utilizing  two  independent  research 


assistants 

from 

the 

Illinois 

State  Police  Research 

and 

Development 

Bureau . 

These 

research 

assistants 

are 

experienced 

at 

administering 

surveys . 

Neither  of 

the 

research  assistants  employed  is  familiar  with  the  technology 
artifact,  and  neither  of  them  supervises  any  of  the 
respondents  involved  in  the  survey.  The  research  assistants 
will  be  coached  by  the  author  on  how  to  conduct  the  survey. 
The  research  assistants  will  provide  the  raw  data  to  the 
author  for  tabulation  and  analysis.  The  author  will  only 
interpret  the  data  as  provided. 

D .  SURVEY 

The  survey  instrument  will  contain  30  closed 

statements.  Six  statements  will  be  used  to  determine  the 


users'  observations  in  each  of  five  areas,  including  ease  of 
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use,  usefulness,  circular  reporting,  complexity  of  data 
sources,  and  information  overload.  No  open  statements  or 
questions  will  be  utilized;  this  will  be  addressed  later  in 
this  chapter  within  the  limitations  section. 

During  the  development  of  the  survey,  the  author  will 
avoid  the  use  of  any  abbreviations,  slang,  or  jargon.  The 
survey  will  be  organized  in  a  manner  that  minimizes  the 
effects  of  both  order  and  context.  Thus,  the  statements 
will  be  in  random  order  so  the  respondent  will  not  answer 
consecutive  statements  from  within  the  same  measurable  area. 
Within  each  area,  half  of  the  statements  will  be  stated  in 
the  negative  and  half  in  the  affirmative  to  prevent  a 
positive  bias.  A  seven  point  Likert  scale  (Strongly  Agree 
to  Strongly  Disagree)  will  be  used  to  measure  variance  in 
respondents'  perceptions. 

Please  see  Appendix  C  for  the  Survey. 

All  survey  items  measuring  a  particular  variable  will 
be  tested  for  Convergent  validity  seeking  Cronbach 1 s  alpha 
reliability  score  of  0.70  or  higher.  Many  social  science 
professionals,  require  a  Cronbach 1 s  alpha  reliability  score 
of  0.70  or  higher.38  "Cronbach 's  alpha  is  viewed  as  a 
measure  of  how  well  the  sum  score  of  the  selected  items 
capture  the  expected  score  for  the  larger  domain,  even  if 
that  domain  is  heterogeneous."  39 


38  J.  Reynaldo  A.  Santos,  "Cronbach' s  Alpha:  A  Tool  for  Assessing  the 
Reliability  of  Scales"  Journal  of  Extension  37,  no.  2  (April  1999) 
http://www.joe.org/joe/1999april/tt3.html  (accessed  January  9,  2008) 

39  Peter  H.  Hopewell,  Assessing  the  Acceptance  and  Functional  Value 
of  the  Asymmetrical  Software  Kit  (ASK)  at  the  Tactical  Level.  (Monterey, 
CA:  Naval  Postgraduate  School,  2007.),  29. 
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All  variables  will  be  tested  for  discriminant  validity 
to  ensure  they  are  moving  separately  and  are  not  measuring 
the  same  thing. 

Although  there  is  no  standard  value  for 
discriminant  validity,  a  result  less  than  .85 
tells  us  that  discriminant  validity  likely  exists 
between  the  two  scales.  A  result  greater  than 
.85,  however,  tells  us  that  the  two  constructs 
overlap  greatly  and  they  are  likely  measuring  the 
same  thing.  Therefore,  we  cannot  claim 
discriminant  validity  between  them.40 

E .  PROCESS 

Prior  to  beginning  the  final  survey  a  Trial  Run  will  be 
performed.  The  Trial  Run  will  be  performed  using  four 
supervisors  of  the  criminal  intelligence  analysts,  all  of 
whom  work  within  the  Statewide  Terrorism  and  Intelligence 
Center.  All  of  the  respondents  who  will  be  involved  in  the 
Trial  Run  will  receive  the  same  training  as  those  who  will 
be  given  the  opportunity  to  participate  in  the  final  survey. 
The  trial  run  will  be  conducted  in  order  to  train  the  team 
and  refine  the  statements.  The  trial  run  will  establish 
that  the  subjects  do  not  have  difficulty  following  the 
directions  and  that  the  process  for  administering  the  survey 
will  function  as  planned.  In  terms  of  the  survey 
instrument,  the  four  respondents  will  complete  the  survey 


40  "Discriminant  Validity" 

http://en.wikipedia.org/wiki/Discriminant  validity  (accessed  January  9, 
2008.  According  to  Wikipedia,  this  information  was  discerned  from  the 
following  sources:  D.  T.  Campbell  and  D.  W.  Fiske,  "Convergent  and 
Discriminate  Validation  by  the  Multitrait-Multimethod  Matrix." 
Psychological  Bulletin  56,  no.  2  (1959)  :  81-105  and  Oliver  P.  John  and 

Veronica  Benet-Martinez ,  "Measurement:  Reliability,  Construct 
Validation,  and  Scale  Construction"  In  Handbook  of  Research  Methods  in 
Social  and  Personality  Psychology,  eds .  Reis,  Harry  T.  Reis  and  Charles 
M.  Judd  (New  York:  Cambridge  University  Press,  2000),  339-369. 
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and  report  if  the  statements  are  understandable  and  lack 
ambiguity.  Feedback  from  the  initial  trial  run  will  be  used 
to  further  develop  the  survey  statements. 

Each  of  the  respondents  will  enter  a  private  office. 
Once  in  the  private  office,  one  of  the  research  assistants 
will  hand  the  survey  directions  (see  Appendix  B)  to  the 
respondent.  The  respondent  will  be  asked  if  he  or  she  has 
any  questions  or  needs  further  clarification  prior  to 
continuing  with  the  survey.  The  research  assistant  will 
answer  the  questions  until  the  respondent  is  prepared  to 
proceed . 

Following  the  completion  of  the  survey,  the  respondent 
will  seal  the  survey  in  an  envelope.  The  respondent  will 
then  exit  the  private  office  and  enter  a  second  private 
office.  Once  in  the  second  private  office  the  respondent 
will  be  greeted  by  the  second  research  assistant.  The 
second  research  assistant  will  inquire  with  the  respondent 
if  he  or  she  understood  the  statements,  and  if  he  or  she  had 
any  concerns  or  thoughts  regarding  the  survey  or  the 
process.  Absent  any  significant  concern,  the  respondent 
will  place  the  sealed  envelope  in  a  box  with  a  cutout  in  the 
top.  This  process  will  insure  the  respondent  that  his  or 
her  responses  will  remain  anonymous  and  confidential.  The 
surveys  will  not  be  collected  from  the  box  until  all 
respondents  have  been  given  the  opportunity  to  participate 
and  have  completed  the  process. 

F.  LIMITATIONS 

There  are  only  34  criminal  intelligence  analysts  who 
work  within  the  Illinois  Statewide  Terrorism  and 
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Intelligence  Center.  Each  of  these  analysts  will  be  given 
the  opportunity  to  participate  in  the  survey.  Due  to  this 
fact,  it  will  not  be  necessary  to  randomly  select 
respondents.  While  all  of  the  respondents  are  being  given 
the  opportunity  to  participate,  the  fact  remains  that  the 
survey  will  include  a  maximum  number  of  30.  Validity  and 
reliability  are  both  maximized  by  the  number  of  respondents 
who  participate.  When  reviewing  the  findings  and 
recommendations  of  this  thesis,  the  number  of  respondents 
will  be  considered. 

The  technology  artifact  being  considered  in  the  survey 
is  a  new  initiative.  The  respondents  were  trained  on  the 
use  of  the  technology  during  two  formal  training  sessions 
both  of  which  occurred  between  November  24  and  December  13, 
2007.  The  training  was  taught  by  both  intelligence 
leadership  from  within  the  Illinois  State  Police  and  the 
company  responsible  for  the  development  of  the  technology 
being  considered.  During  the  training  each  of  the 
respondents  completed  the  same  training  exercises  and 
scenarios,  thus  they  have  a  similar  level  of  baseline 
experience.  Each  of  the  respondents  has  used  the  technology 
artifact  in  the  regular  course  of  their  business  for  a 
minimum  of  five  work  shifts.  When  reviewing  the  findings 
and  recommendations  of  this  thesis  the  level  of  experience 
of  the  respondents  will  be  considered. 

The  respondents  have  anticipated  the  delivery  of  this 
technology  artifact  since  April  2007.  Due  to  internal 
budgetary  issues  and  internal  network  problems  within  the 
Illinois  State  Police,  the  company  responsible  for  the 
delivery  of  the  technology  artifact  was  unable  to  install 
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the  system.  This  delay  was  a  source  of  frustration  for 
several  of  the  respondents.  It  is  unclear  how  this 
frustration  may  impact  the  perception  of  the  respondents; 
nonetheless,  this  will  be  considered  when  reviewing  the 
findings . 

As  previously  stated  within  this  chapter,  the  survey 
will  consist  of  closed  statements.  The  use  of  only  closed 
statements  may  limit  the  type  of  responses  from  the 
respondents.  The  respondents  will  not  be  able  to  provide 
any  details  regarding  the  reason,  thought  process,  or  frame 
of  reference  for  the  answers  provided.  Another  limitation 
to  the  closed  statement  format  is  that  the  author  will  be 
unable  to  understand  or  explain  unanticipated  findings. 
When  reviewing  the  findings  and  recommendations  of  this 
thesis  the  type  of  statements  posed  during  the  survey  will 
be  considered. 
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IV.  SURVEY  FINDINGS 


A.  INTRODUCTION 

The  results  of  the  survey  completed  by  the  respondents 
were  tabulated  and  analyzed  using  SPSS  15.0.  Many  of  the 
items  in  this  survey  were  drawn  from  previously  validated 
instruments,  specifically,  the  Technology  Acceptance  Model 
(TAM)  ,  studies  conducted  by  Lin,  HU  and  Chen41  as  well  as 
previous  research  done  by  Lin42,  and  subsequent  research  by 
Hopewell43.  The  type  of  survey  statements  were  extracted 
from  these  bodies  of  works  and  then  altered  to  meet  the 
demands  of  this  particular  study.  In  addition  to  these 
instruments,  the  Senior  Intelligence  Analyst  from  the 
Illinois  State  Police  and  a  retired  Illinois  State  Police 
Major  with  extensive  intelligence  and  information  technology 
experience  were  called  upon  to  validate  the  statements. 

This  study  examines  the  instrument's  convergent  and 
discriminate  validity.  Using  the  responses  from  the  survey, 
the  instrument's  convergent  validity  was  examined  using  the 
Cronbach' s  alpha  value  for  each  of  the  specified  constructs. 
As  previously  stated,  many  social  science  professionals 
require  a  Cronbach' s  alpha  reliability  score  of  0.70  or 

41  Paul  Jen-Hwa  Hu,  Chienting  Lin  and  Hsinchun  Chen,  "User  Acceptance 
of  Intelligence  and  Security  Informatics  Technology:  A  Study  of 
COPLINK."  Journal  of  the  American  Society  for  Information  Science  and 
Technology  56,  no.  3  (2005)  :  235-244. 

42  Chienting  Lin,  "Examining  Technology  Usability  and  User  Acceptance 
in  Digital  Government:  A  Case  Study  in  Law  Enforcement"  (PhD,  Pace 
University),  2004. 

43  Peter  H.  Hopewell,  Assessing  the  Acceptance  and  Functional  Value 
of  the  Asymmetrical  Software  Kit  (ASK)  at  the  Tactical  Level.  (Monterey, 
CA:  Naval  Postgraduate  School,  2007.) 
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higher.44  The  results  of  the  reliability  analysis  are 
displayed  in  the  tables  throughout  this  chapter.  The 
Cronbach' s  Alpha  values  ranged  between  .651  and  .848. 
Although  not  all  values  met  the  .70  or  higher  preferred 
reliability  score,  most  exceeded  and  those  that  did  not  were 
relatively  close. 

The  survey  was  completed  by  thirty  intelligence 
analysts  referred  to  throughout  this  chapter  as  respondents. 
Four  of  the  thirty-four  analysts  who  worked  at  the  STIC  were 
unavailable.  Each  of  the  respondents  surveyed  worked  at  the 
Illinois  Statewide  Terrorism  and  Intelligence  Center  (STIC) 
located  in  Springfield,  Illinois.  The  respondents 

represented  the  Illinois  State  Police,  Illinois  National 
Guard,  the  Federal  Bureau  of  Investigation,  the  Drug 
Enforcement  Administration,  and  the  Department  of  Homeland 
Security.  Each  of  the  respondents  is  well  educated  and 
possessed  a  minimum  of  a  bachelor's  degree.  Eighteen  of  the 
respondents  had  more  than  3  years  of  analytical  experience. 
All  were  proficient  in  the  used  intelligence-related 
technology  tools  such  as,  the  Law  Enforcement  Data  System, 
Violent  Crime  Information  Tracking,  and  Linking  System, 
Chicago  Area  Crime  Database,  Regional  Data  Exchange,  and 
numerous  others . 

All  thirty  of  the  respondents  agreed  in  writing  to 
participate.  An  independent  research  team  administered  the 
survey,  and  each  respondent  was  assured  both  verbally  and 
through  the  process  that  their  responses  would  be  anonymous. 
Refer  to  Chapter  III  for  further  details  regarding  the 

44  J.  Reynaldo  A.  Santos,  "Cronbach' s  Alpha:  A  Tool  for  Assessing  the 
Reliability  of  Scales"  Journal  of  Extension  37,  no.  2  (April  1999) 
http://www.joe.org/joe/1999april/tt3.html  (accessed  January  9,  2008) 
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methodology  process  utilized.  It  should  be  noted  that  three 
of  the  thirty  responses  were  not  considered  due  to  being 
incomplete  or  non-responsive . 

This  chapter  presents  a  detailed  examination  of  five 
hypotheses : 

1.  Information  overload  exists  within  the  STIC. 

2.  Circular  reporting  is  occurring. 

3.  Complexity  of  data  sources  has  a  negative  effect 
on  the  efficiency  and  effectiveness  of  the 
analysts  within  STIC. 

4 .  A  context-based  technology  product  has  reduced 
circular  reporting. 

5.  A  context-based  technology  product  within  the  STIC 
has  assisted  with  the  management  of  complex  data 
sources . 

The  survey  instrument  used  a  seven  point  Likert  scale, 
where  a  neutral  mean  value  (e.g.,  4)  was  neither  strong  nor 

weak.  For  purposes  of  analysis,  ten  negatively  worded 
statements  were  used  to  test  for  a  positive  bias  in  the 
survey  responses.  Due  to  the  need  to  aggregate  both  the 
negatively  and  positively  worded  statements,  an  adjusted 
mean  had  to  be  utilized.  This  adjusted  mean  takes  into 
account  the  fact  that  the  responses  to  the  negatively  coded 
statements  were  inverted.  Accordingly,  a  positively  coded 
statement  used  the  following  Likert  scale  (l=Strongly  Agree, 
2=Agree,  3=Somewhat  Agree,  4=Undecided,  5=Somewhat  Disagree, 
6=Disagree,  7=Strongly  Disagree)  and  the  negatively  coded 
statements  used  the  following  Likert  scale  (l=Strongly 
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Disagree,  2=Disagree,  3=Somewhat  Disagree,  4=Undecided, 
5=Somewhat  Agree,  6=Agree,  7=Strongly  Agree) . 

In  each  of  the  tables  below,  the  survey  statements 
with  an  adjusted  mean  value  (1.0-3. 9)  shows  positive 
attributes,  whereas  an  adjusted  mean  value  (4. 1-7.0)  shows 
negative  attributes.  As  in  the  Hopewell  research,  "multiple 
questions  were  used  to  measure  single  variables,  allowing 
for  variation  in  how  individuals  respond  to  varying  word 
structures . "45 

As  the  title  of  this  thesis  indicates,  the  author 
attempted  to  answer  the  issue:  Does  the  impact  of  a 
"context-based  approach"  to  the  design  of  tools  for 
intelligence  analysts  mitigate  information  overload? 
Furthermore,  the  paper  was  founded  on  the  premise  that 
information  overload  is  negatively  affected  by  both  circular 
reporting  and  the  complexity  of  data  sources.  In  an  effort 
to  substantiate  this  theory,  the  survey  was  designed  to 
capture  the  self-reported  perception  of  the  respondents 
regarding  the  effect  of  the  context-based  approach  to  the 
development  of  technology  on  each  of  these  areas. 

B.  INFORMATION  OVERLOAD 

The  issue  of  information  overload  was  examined  in  two 
distinct  ways:  1)  do  the  respondents  perceive  that 
information  overload  exists  within  the  STIC;  2)  what  affect 
do  respondents  perceive  the  context-based  technology 
artifact  has  on  the  information  overload  issue? 


45  Peter  H.  Hopewell,  Assessing  the  Acceptance  and  Functional  Value 
of  the  Asymmetrical  Software  Kit  (ASK)  at  the  Tactical  Level  (Monterey, 
CA:  Naval  Postgraduate  School,  2007),  27. 
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1. 


Is  there  a  perception  that  Information  Overload 
exists  within  the  STIC? 


Two  statements  were  included  in  the  survey  about  the 
perception  of  the  respondents  regarding  information  overload 
at  STIC. 


Statement 

# 

Summary  of  Survey 
Statement 

Adjusted 

Mean 

Standard 

Deviation 

8 

Amount  of  information 
available  to  me  is 
difficult  to  handle. 

3.96 

1 . 87 

26 

Amount  of  information  I 
am  expected  to  review 
makes  it  difficult  for  me 
to  complete  strategic 
analysis . 

2 . 93 

1.51 

Table  3.  Survey  Statements  Regarding  Information  Overload  within 

STIC 

These  two  statements  were  combined  into  one  index  and 
analyzed  together,  the  results  are  found  below. 


Statements 

Adjusted  Mean  of 
the  Index  for 
Information 

Overload  at  STIC 

Cronbach' s 
Alpha 

Range 

Standard 

Deviation 

8  &  26 

3.4 

.747 

1-6.5 

1.52 

Table  4.  Index  for  Information  Overload  at  STIC 


Upon  close  examination  of  the  standard  deviation  and 
range  data  one  finds  that  the  respondents  did  not  agree  that 
information  overload  existed  within  the  STIC.  This 
disagreement  led  to  an  index  mean  of  3.4.  This  score  has  a 
tendency  to  reflect  that  there  was  a  preponderance  of 
support  indicating  that  the  respondents  leaned  toward  the 
belief  that  information  overload  existed. 
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2 .  Perception  of  the  affect  that  the  context-based 
technology  approach  will  have  on  information 
overload  within  the  STIC? 

Four  statements  were  included  in  the  survey  regarding 
the  respondents'  perception  regarding  the  affect  that  the 


context-based  technology  approach  had  on  information 
overload  at  STIC. 


Statement 

# 

Summary  of  Survey  Statement 

Adjusted 

Mean 

Standard 

Deviation 

3 

Context-based  technology 
approach  will  increase  the 
time  I  spend  collating 
information . 

2.55 

1 . 15 

4 

The  context-based  technology 
approach  will  improve  my 
ability  to  handle 
information . 

2.25 

.  944 

11 

Use  of  a  context-based 
technology  approach  will 
decrease  the  time  I  spend 
collating  information. 

2 .77 

1.08 

28 

Use  of  a  context-based 
technology  approach  will 
reduce  the  time  I  spend 
collecting  information. 

2 .44 

.  974 

Table  5.  Survey  Statements  Regarding  Context-Based 
Technology  Effect  on  Information  Overload 


These  four  statements  were  combined  into  one  index  and 


analyzed  together,  the  results  are  found  below. 


Statements 

Adjusted  Mean  of  the 
Index  for  the  Effect 

of  Context-Based 
Approach  on 
Information  Overload 

at  STIC 

Cronbach' s 
Alpha 

Standard 

Deviation 

3,4,11,28 

2.50 

.737 

.779 

Table  6.  Index  for  Effect  of  Context-Based  Approach  on 

Information  Overload  at  STIC 
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In  looking  at  the  data  there  appears  to  be  strong 
agreement  that  the  context-based  technology  product  analyzed 
in  this  study  had  a  positive  impact  on  the  information 
overload  at  the  STIC.  This  is  an  important  finding.  If  we 
couple  this  with  the  findings  discussed  above  regarding  the 
perceived  level  of  information  overload  at  the  STIC,  one  can 
reason  that  the  context-based  product  studied  has  reduced 
current  information  overload  and  may  be  beneficial  in  the 
future  as  additional  data  sources  are  made  available  to  the 
respondents.  Regardless  of  the  perceived  current  level  of 
information  overload,  it  is  clear  the  respondents  feel  the 
product  examined  has  allowed  them  to  handle  information  more 
effectively  and  more  efficiently. 

C .  CIRCULAR  REPORTING 

The  issue  of  circular  reporting  was  examined  in  two 
distinct  ways.  First,  did  the  respondents  perceive  that 
circular  reporting  exists?  Second,  what  affect  did  the 
respondents  perceive  the  context-based  technology  artifact 
had  on  the  circular  reporting  issue? 

1.  Is  there  a  perception  that  circular  reporting 
exists? 

Three  statements  were  included  in  the  survey  regarding 
the  perception  of  the  respondents  regarding  circular 
reporting  at  STIC. 
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Statement 

# 

Summary  of  Survey  Statement 

Adjusted 

Mean 

Standard 

Deviation 

10 

Productivity  is  negatively 
impacted  by  the  amount  of 
times  I  read  duplicate 
material . 

2.70 

1.29 

17 

Seldom  am  I  expected  to  read 
duplicate  information. 

2.51 

1.08 

22 

Frequently  I  am  expected  to 
read  duplicate  information. 

2.70 

1.23 

Table  7 .  Survey  Statements  Regarding  Circular  Reporting 

Within  STIC 


These  three  statements  were  combined  into  one  index  and 
analyzed  together,  the  results  are  found  below. 


Statements 

Mean  of  the  Index  for 
Circular  Reporting  at 
STIC 

Cronbach' s 
Alpha 

Standard 

Deviation 

10, 17,22 

2 . 64 

.  669 

.  937 

Table  8 .  Index  for  Circular  Reporting  at  STIC 


A  review  of  the  data  indicates  there  is  agreement  that 
circular  reporting  existed  (e.g.,  2.64) .  A  review  of  this 

data  shows  a  few  respondents  who  disagreed  with  this 
premise . 

2.  Perception  of  the  affect  that  the  context-based 
technology  approach  will  have  on  circular 
reporting? 

Four  statements  were  included  in  the  survey  regarding 
the  respondents'  perception  about  the  affect  that  the 
context-based  technology  approach  had  on  circular  reporting 
at  STIC. 
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Statement 

# 

Summary  of  Survey  Statement 

Adjusted 

Mean 

Standard 

Deviation 

6 

Context-based  technology 
will  increase  the  number  of 
times  I  read  duplicate 
information . 

3.37 

1.49 

23 

Use  of  a  context-based 
technology  approach  will 
reduce  the  number  of  times  I 
am  required  to  read 
duplicate  information. 

2.96 

1 .19 

Table  9.  Survey  Statements  Regarding  Context-Based  Technology 

Effect  on  Circular  Reporting  within  STIC 

These  two  statements  were  combined  into  one  index  and 
analyzed  together,  the  results  are  found  below. 


Statements 

Adjusted  Mean  of  the 
Index  for  the  Effect 

of  a  Context-Based 
Approach  on  Circular 
Reporting 

Cronbach' s 
Alpha 

Standard 

Deviation 

6  &  23 

3.16 

.  695 

1 .18 

Table  10.  Index  for  Effect  of  Context-Based  Approach 

on  Circular  Reporting  within  STIC 


In  reviewing  the  findings  of  this  particular  construct 
it  was  clear  the  respondents  believed  there  were  positive 
attributes  to  how  the  context-based  product  that  was 
examined  effected  circular  reporting. 

D.  COMPLEX  DATA  SOURCES 

The  issue  of  complex  data  sources  was  examined  in  two 
distinct  ways.  First,  did  the  respondents  perceive  that  the 
issue  of  complex  data  sources  was  problematic  within  the 
STIC?  Secondly,  what  affect  did  the  respondents  perceive 
the  context-based  technology  artifact  had  on  the  complex 
data  sources  issue? 
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1. 


Is  there  a  perception  that  complex  data  sources 
cause  a  negative  impact  within  the  STIC? 


Two  statements  were  included  in  the  survey  regarding 
the  perception  of  the  respondents  regarding  complex  data 
sources  at  STIC. 


Statement 

# 

Summary  of  Survey  Statement 

Adjusted 

Mean 

Standard 

Deviation 

15 

The  number  of  data  sources 

I  am  required  to  check 
makes  it  difficult  for  me 
to  provide  a  timely 
response . 

2 . 92 

1 .41 

30 

Number  of  data  sources 
available  to  me  is  easy  to 
manage . 

3.70 

1.48 

Table  11.  Survey  Statements  Regarding  Complex  Data 

Sources  within  STIC 


These  two  statements  were  combined  into  one  index  and 
analyzed  together,  the  results  are  found  below. 


Statements 

Adjusted  Mean  of  the 
Index  for  the  Issue  of 
Complex  Data  Sources 
at  STIC 

Cronbach' s 
Alpha 

Standard 

Deviation 

15  &  30 

3.31 

.  651 

1.24 

Table  12.  Index  for  Complex  Data  Sources  at  STIC 


A  review  of  the  data  shows  disagreement  amongst  the 
respondents.  Although,  most  believed  that  the  data  sources 
available  within  STIC  had  a  negative  impact,  others  believed 
that  the  sources  available  were  manageable.  The  index  mean 
of  3.31  indicates  there  is  moderate  concern  regarding  this 
facet  of  information  overload. 
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2 .  Perception  of  the  affect  that  the  context-based 
technology  approach  will  have  on  complex  data 
sources  within  the  STIC? 

Three  statements  were  included  in  the  survey  regarding 
the  respondents'  perception  regarding  the  affect  that  the 
context-based  technology  approach  had  on  complex  data 
sources  at  STIC. 


Statement 

# 

Summary  of  Survey  Statement 

Adjusted 

Mean 

Standard 

Deviation 

7 

Context-based  technology 
approach  will  make  it  more 
difficult  to  handle  the 

number  of  data  sources 
available  to  me. 

2.33 

1 . 10 

16 

The  context-based  technology 
approach  will  make  it  easier 
to  handle  the  number  of  data 

sources  available  to  me. 

2.25 

.784 

21 

It  would  be  beneficial  to  me 
if  more  data  sources  were 
accessible  through  a 
context-based  technology 
approach . 

1.62 

.  966 

Table  13.  Survey  Statements  Regarding  Context-Based 

Approach  on  Complex  Data  Sources 


These  three  statements  were  combined  into  one  index  and 
analyzed  together.  The  results  are  found  below. 


Statements 

Adjusted  Mean  of  the 
Index  for  the  Effect 
of  Context-Based 
Technology  on  Handling 
Complex  Data  Sources 

Cronbach' s 
Alpha 

Standard 

Deviation 

7,16,21 

2.09 

.700 

.761 

Table  14.  Index  for  Effect  of  Context-Based  Approach 


on  Complex  Data  Sources  within  STIC 
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The  fact  that  the  respondents  strongly  (mean  of  2.09) 
believed  that  the  context-based  technology  product  had  a 
positive  impact  on  how  they  handle  complex  data  sources  is 
an  important  finding. 

E.  SUMMARY  OF  FINDINGS 

Several  key  findings  are  noted  in  the  analysis  outlined 
above.  Below  is  a  summary  of  those  critical  observations. 

1.  A  comparison  of  the  indexes  involving  the  current 
perceived  situation  at  the  STIC  reflects  that  the 
respondents  believed  circular  reporting  was  the  most 
significant  concern  (mean  of  2.64),  followed  by 
complex  data  sources  (mean  of  3.31)  .  Finally,  the 
respondents  viewed  information  overload  is  the  least 
concern  of  the  three  areas  studied  (mean  of  3.4)  . 

2.  Of  interest  is  the  finding  that  both  circular 
reporting  and  complex  data  sources  were  viewed  to  be 
of  greater  concern  than  the  issue  of  information 
overload.  This  finding  begs  the  question,  how  could 
the  respondents  view  both  circular  reporting  and 
complex  data  sources  to  be  of  more  significant 
concern  than  the  more  general,  overarching  issue  of 
information  overload? 

3.  In  each  of  the  areas  examined,  information  overload, 
circular  reporting,  and  complex  data  sources,  the 
respondents  believed  the  context-based  system 
examined  can  improve  the  efficiency  and  effectiveness 
of  the  current  state  of  being. 

4 .  The  respondents  reported  that  the  context-based 

product  had  the  greatest  impact  on  the  handling  of 
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complex  data  sources  (mean  of  2.09),  followed  by  the 
ability  to  minimize  information  overload  (mean  of 
2.5)  .  Although  the  respondents  believed  the  system 
had  a  positive  impact  on  circular  reporting  (mean 
3.4),  they  believed  the  impact  would  not  be  as  great 
as  that  of  the  other  two  constructs. 


67 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


68 


V.  RESEARCH  OBSERVATIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

As  new  concepts  have  been  conceived,  engineered,  and 
executed  throughout  history,  so  too  has  the  conventional 
wisdom  to  seek  out  specialists  who  can  assist  in  the  pursuit 
of  constructing  the  conceptualizations  of  these  idea 
generators,  or  architects.  The  ability  to  tap  into 
specialized  knowledge  provides  for  on-demand,  content- 
specific  expertise  to  be  applied  to  the  corresponding  issue 
or  problem.  This  "specialist"  generally  seeks  to  understand 
the  circumstances  of  the  assigned  task,  formulate  a  plan 
based  upon  his  or  her  assessment  and  pre-existing  knowledge, 
make  predictions  about  how  to  affect  or  move  the  task  to  the 
desired  state,  put  those  plans  into  action,  proactively 
monitor  the  affects  from  the  action  and  the  overall  solution 
-  making  adjustments  as  necessary  and  repeating  the  process 
until  a  final,  hopefully  successful,  state  is  reached. 

With  the  perspective  that  "context"  is  core  to  deeper 
understanding  of  issues,  the  Illinois  State  Police  in  2005 
entered  into  an  agreement  with  RiverGlass,  Inc.  to  develop  a 
context-based  technology  solution.  The  system  was  designed 
in  association  with  intelligence  analysts  at  the  Illinois 
Statewide  Terrorism  and  Intelligence  Center  (STIC) .  What 
separated  this  system  from  others  was  the  fact  that  it  was 
designed  to  operate  akin  to  an  intelligence  analyst; 
fundamentally  the  context  of  how  the  end  user  would  operate 
using  this  system  was  the  driving  principle  throughout  the 
project  lifecycle.  Accordingly,  the  system  was  built  to 
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intuitively  operate  and  process  information  comparably  to 
independent,  intelligence  analysts. 

Throughout  the  development  of  this  context-based 
technology,  numerous  municipal,  state,  and  Federal  law 
enforcement,  intelligence,  and  military  entities  contacted 
the  Illinois  State  Police  to  gain  an  understanding  of  the 
scope  of  this  endeavor.  Inevitably  questions  were  raised, 
for  instance:  1)  Does  the  context-based  approach  reduce 
information  overload?  2)  Does  the  context-based  approach 
reduce  circular  reporting?  3)  Can  the  context-based 
approach  combine  both  structured  and  unstructured  data  from 
a  variety  of  sources  and  types,  and  thus,  reduce  the 
complexity  of  data  sources?  At  the  time,  due  to  the  stage 
of  development  of  this  technology  the  Illinois  State  Police 
discovered  it  was  not  possible  to  definitively  answer  any  of 
these  questions;  that  deficiency  prompted  this  research. 

The  research  in  this  thesis  has  established  that  the 
existence  of  information  overload,  circular  reporting,  and 
the  complexity  of  data  sources  negatively  impacts  the 
performance  of  intelligence  analysts  who  work  within  fusion 
centers.  Furthermore,  the  research  has  established  that 
intelligence  analysts  perceive  the  use  of  a  context-based 
technology  solution  will  improve  their  efficiency  and 
effectiveness  as  well  as  the  overall  operation  of  the  STIC. 

Based  on  these  findings,  several  specific 
recommendations  have  emerged;  these  recommendations  are 
found  below.  In  addition,  gaps  in  the  literature  are 
recognized.  Unanswered  questions  stemming  from  this 
research  have  also  materialized  and  will  be  introduced. 
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It  is  clear  that  in  advance  of  making  specific 
recommendations,  they  should  be  aimed  at  accomplishing  the 
following  goals. 

1 .  Circular  Reporting 

•  Identify  key  strategies  for  reducing  the 
production  of  circular  reports. 

•  Identify  mechanisms  for  detecting  and  dealing 
with  duplicate  input  as  early  as  possible  in 
fusion  center  operations. 

2 .  Complexity  of  Data  Sources 

•  Minimize  the  complexity  of  accessing  and 
utilizing  multiple  data  sources. 

•  Maximize  the  utility  of  all  known  information 
in  fulfilling  the  mission  of  the  intelligence 
community . 

•  Reduce  the  time  it  takes  for  analysis  and 
simultaneously  increase  the  comprehensiveness 
of  the  analysis  output. 

•  Enable  and  promote  both  inter-  and  intra¬ 
entity  synthesis,  not  simply  the  sharing  of 
information . 

B .  RECOMMENDATIONS 

1 .  Leadership 

Research  Observation :  Information  overload  issues  are 

both  real  and  inevitable.  A  fundamental  question  is,  "What 
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are  the  plans  for  dealing  with  an  information  dense 
environment  wherein  the  data  continues  to  grow 
exponentially?"  It  is  clear  that  the  intelligence 

community,  e.g.,  fusion  centers,  cannot  solve  this  issue  by 
simply  hiring  additional  intelligence  analysts.  Leaders  of 
these  agencies  must  recognize  and  support  the  fact  that  a 
new  solution  _i_s  required. 

As  technology  grew,  culminating  in  the  "dot-com"  boom 
of  2000,  most  agencies'  information  technology  (IT) 
departments  developed  an  over- inf lated  perception  of  their 
role.  This  manifested  itself  in  IT  specialists  taking  on 
too  much  authority  in  directing  key  strategic  projects  for 
which  they  were  ill -equipped  to  thoroughly  comprehend,  much 
less  lead.  Business  partners  acquiesced  to  IT  specialists' 
recommendations,  as  they  did  not  understand  technology  well 
enough  to  build,  manage,  or  deploy  a  technology  product. 

Gradually,  IT's  business  partners  came  to  the  realization 
that  IT  is  a  support  function  or  tool,  just  as  human 
resources  and  administrative  functions  are,  not  a  profit 
center.  Albeit,  IT  is  integral  to  business  solutions  but  it 
should  not  have  control  of  the  strategic  direction.  Its 
role  is  one  of  enabler,  delivering  technology  solutions  that 
are  more  effective  and  efficient,  in  accordance  and  in 
partnership  with  their  business  partners.  Unfortunately, 
leagues  of  mid-  to  senior- level  managers  running  IT 
departments  have  not  embraced  this  approach  to  strategy 
development . 

Recommendation :  Strong  leadership  must  emerge  from 

both  business  partners  and  IT  specialists  in  order  to  solve 

the  challenges  of  circular  reporting  and  complexity  of  data 

sources.  Both  must  re-engineer  their  approach  to  resolving 

technology  problems.  As  "end  users"  identify  deficiencies 

that  require  a  technology  solution,  IT  specialists  must 
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partner  with  them  by  leveraging  their  expertise.  The  IT 
professional  must  draw  out  of  the  end  user  the  scope  of  the 
deficiency,  compliment  that  with  their  understanding  of  the 
domain,  and  facilitate  the  development  and  implementation  of 
"context-based"  technology  tools. 

At  the  same  time,  technology  companies  need  to  invest 
resources  on  the  front  end  of  the  issue  to  develop  a  deep 
understanding  of  the  problem  space.  It  is  through  this 
level  of  understanding  that  rich  solutions  will  be 
developed,  produced,  and  implemented. 

2 .  Circular  Reporting 

a.  Awareness  and  Focus 

Research  Observation:  Today's  intelligence 
environment  is  a  distributed  network  comprised  of 
autonomous,  loosely  coupled  nodes,  each  node  being  a 
Federal,  state,  or  municipal  intelligence/fusion/operations 
center.  This  environment,  in  conjunction  with  information 
sharing  imperatives,  complicates  the  issue  of  circular 
reporting.  The  network  nodes  have  different  missions, 
operate  differently,  and  have  different  sources  of 
information;  yet  they  are  increasingly  expected  to  share 
information  with  each  other.  While  this  lends  itself  to  an 
information  dense  environment,  the  likelihood  of  duplicity 
and  redundancy  is  all  too  probable.  The  issue  is  compounded 
by  the  presumption  that  this  intelligence  environment  works 
with  a  high  degree  and  mixture  of  collaborative 
interdependence  and  autonomy.  It  is  with  this  increased 
awareness  that  a  solution  should  be  advanced. 
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Recommendation:  Establish  within  each  fusion 
center  a  work  team  focused  on  circular  reporting.  This  team 
should  develop  procedures  and  guidelines  aimed  at  reducing 
circular  reporting. 

b.  Best  Practices 

Research  Observation :  The  problem  of  circular 
reporting  and  management  of  redundant  information  persists 
in  many  domains.  Two  are  relevant  to  fusion  centers  -- 
patent  processing  and  education. 

The  job  of  the  patent  office  is  to  take  every  new 
submission  and  ensure  that:  a)  it  does  not  infringe  on 
another  patent  (e.g.,  it  is  not  duplicative),  and  b)  it  is 
new  and  novel  enough  to  receive  patent  protection.  This 
implies  a  new  submission  is  compared  to  all  existing  patents 
at  an  early  stage.  It  further  implies  a  fairly  deep 
understanding  of  the  content  of  the  submission  in  order  to 
assess  the  novelty  of  the  idea.  The  key  ideas  here  are  that 
the  patent  office  has  a  formal  structured  process  for 
receiving  submissions  (e.g.,  intake)  and  a  separate 
analytical  step  wherein  the  content  of  the  submission  is 
compared  to  pre-existing  knowledge  bases.  In  some  ways  the 
patent  office  is  analogous  to  any  single  node  in  the  law 
enforcement  information-sharing  network. 

The  educational  domain  also  has  a  mechanism  of 
minimizing  duplicate  information  reporting.  In  fact,  this 
thesis  makes  diligent  use  of  it  -  the  bibliographic 
reference.  The  bibliography  references  the  sources  utilized 
in  the  creation  of  new  knowledge  or  information.  This 


74 


social  protocol  is  widely  adopted  and  utilized  in  the 
production  of  most  formal  documents. 

Lastly,  another  process  exists  within  the  fusion 
center  environment  which  focuses  less  on  technology  and  more 
on  human  intervention.  This  best  practice  is  known  as  the 
Threat  Identification  Group  (TIG) .  The  real  function  of  the 
TIG  is  to  synthesize  information  across  individuals  and 
specialties  on  a  daily  basis.  It  is  during  these  daily 
meetings  that  the  complexity  of  cross-domain  tasks,  time 
constraints,  and  unknowns  are  surfaced  in  order  to  make 
decisions  and  set  tactical  direction  based  on  the  totality 
of  what  is  known,  and  as  important,  what  is  needed,  e.g., 
human  intuition,  and  the  best  most  likely  path  toward  the 
desired  outcome  at  that  time.  It  is  through  this  type  of 
human  interaction  that  real  "dots"  can  be  identified, 
understood,  supplemented,  and  ultimately  connected;  which  is 
what  humans  excel  at. 

The  role  of  technology  is  to  serve  as  an 
"electronic  staff  officer"  for  the  analyst,  for  instance, 
knowing  what  sources  and  content  are  available,  knowing  who 
and  how  to  contact  resources  (content  and  human) , 
understanding  the  intent  of  the  analyst  as  he\she  "works 
his\her  way"  toward  a  conclusion,  knowing  the  techniques  and 
processes  that  are  most  likely  to  bring  success. 

Recommendation:  Investigate  successful  models 
that  are  already  addressing  the  broad  issue  of  information 
overload  and  the  specifics  of  circular  reporting  and 
complexity  related  to  data  sources. 
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c. 


Structure  and  Operations 


Research  Observation :  Most  fusion  centers  today 

operate  either  implicitly  or  explicitly  on  a  model  of 
gathering  as  much  input  as  possible,  performing  analysis  on 
the  obtained  information,  and  then  sharing  the  analytical 
findings  with  other  entities.  This  Intake  >  Analyze  > 
Disseminate  model  is  conducive  to  both  understanding  and 
combating  circular  reporting.  The  illustration  below 

depicts  this  structure. 


Disseminate 


Analyze 


Intake 
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Model 

Component 

Description 

Intake 

Techniques  applied  to  prevent  duplicate 
information  caused  by  circular  reporting 
from  external  entities  from  entering  the 
fusion  center.  It  is  the  first  point  in 

the  model  where  detection  of  circular 
reporting  is  possible. 

Analyze 

Dealing  with  redundant  information 
contained  within  information  sources. 
This  thesis  elaborated  on  the  application 
of  context-based  technology  in  this  area. 

Disseminate 

Analytical  and  informational  output  is 
shared  with  other  fusion  center  partners. 
It  is  the  point  in  the  model  that  focuses 
on  preventing  the  creation  of  circular  and 
redundant  information. 

Table  15.  Intake,  Analyze,  Disseminate  Descriptions 


Recommendation:  Explore  and  validate  the 
relevancy  of  an  intake,  analyze,  and  disseminate  model  with 
other  fusion  centers.  By  engaging  other  centers  and  perhaps 
partnering  with  them,  further  advance  the  concept  of  a 
similar  operational  models  that  enable  the  sharing  of 
information.  More  specifically,  the  "Intake"  module  applies 
techniques  to  prevent  duplicate  information  caused  by 
circular  reporting  from  entering  the  intelligence  center  at 
all.  The  "Analyze"  module  utilizes  a  variety  of  analytical 
techniques  and  practices  to  comprehend  and  "make  sense"  out 
of  the  situation  at  hand  for  the  purpose  of  making  decisions 
and  taking  action.  The  "Disseminate"  module  ensures  that 
information  is  shared  with  only  those  partners  whom  were 
previously  agreed  to,  but  never  redundantly. 
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d.  Survey  of  Technologies 

Research  Observation :  The  problems  associated 
with  circular  reporting  seem  to  advance  once  a  document 
arrives  at  its  intended  destination.  It  should  be  possible 
to  inspect  inbound  items  and  determine  if  the  content  is  new 
and  relevant  or  redundant  and  irrelevant.  While  the  process 
is  not  a  simple  mechanical  one,  in  that  it  contains  the 
complexity  and  ambiguity  of  language,  it  is  also  not  beyond 
the  reach  of  our  current  technology  aids.  This  is  not  to 
say  that  technology  could  or  should  make  decisions 
autonomously  about  the  validity  of  an  intake  submission,  but 
that  it  could  aid  and  guide  the  human  operator  in  making  the 
determination.  A  brief  review  of  techniques  for  identifying 
duplicate,  or  similar  documents,  vary  greatly  in  approach 
and  sophistication.  Other  domains,  such  as  duplication  from 
the  archival  industry,  have  emerged  over  the  past  several 
years  that  may  hold  promise.  Likewise,  techniques  including 
semantic  and  cluster  analysis  have  also  evolved. 

Recommendation:  A  more  formal  survey  of  duplicate 
detection  as  it  relates  to  circular  reporting  in  the 
intelligence  and  fusion  center  business  should  be 
undertaken.  Finding  a  technology  that  would  include 
artificial  intelligence  and  learning  algorithms  that 
semantically  analyzes  the  concepts  of  the  document  and 
possibly  categorizes  some  into  a  domain  taxonomy  or  ontology 
would  be  of  great  value.  The  ideal  situation  would  be  to 
mix  and  match  these  technologies,  outfit  the  resulting  tool 
with  a  user  interface  specifically  designed  for  the  intake 
function  of  a  fusion  center,  and  embed  it  in  the  work 
environment  of  the  person  performing  the  intake. 
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e .  Feedback  Procedures 

Research  Observation:  The  impetus  to  share 
information  in  today' s  intelligence  communities  without 
clear  guidance,  policies,  and  shared  strategies,  while 
having  had  some  success,  has  really  only  exacerbated  the 
circular  reporting  issue.  More  duplicate  and  redundant 
information  than  ever  has  to  be  sifted  through.  This 
information  is  usually  generated  by  another  fusion  center. 
The  overhead  and  resources  required  to  deal  with  the  inflow 
leaves  little  time  to  focus  on  techniques  and  procedures  for 
reducing  the  onslaught. 

Recommendation:  Establish  a  shared  procedure  for 
communication  to  the  originator  upon  receipt  of  redundant 
information.  For  example,  if  fusion  center  A  and  B  both 
receive  information  from  DHS  and  fusion  center  B  also 
redistributes  to  fusion  center  A,  then  fusion  center  A 
should  have  both  a  means  and  a  responsibility  to  communicate 
to  fusion  center  B  feedback  about  the  redundant  information. 

f.  Dissemination  Authority 

Research  Observation:  The  decision  made  to 
disseminate  information  is  typically  made  by  humans.  One  of 
the  problems  in  fusion  center  operations  is  the  lack  of  a 
known,  named  person,  or  role  responsible  for  the  publishing 
of  information  products  by  the  center.  In  the  worst  cases, 
dissemination  is  left  up  to  each  analyst.  In  other  public 
safety  domains  there  exists  a  role  responsible  for 
coordinating  and  collaborating  with  external  entities  (e.g.. 
Public  Information  Officer  (PIO) ,  corporate  liaisons,  etc.). 
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This  construct  is  worthy  of  investigation  to  advance  its 
feasibility  and  applicability  to  reducing  the  production  of 
circular  reporting. 

Recommendation:  Investigate  the  merits  of 
creating  a  PIO  equivalent  within  each  fusion  center  that  is 
responsible  for  disseminating  information  that  originates 
from  that  fusion  center.  The  list  of  PIOs  should  be  made 
available  to  all  participants  within  the  information-sharing 
network.  Should  the  concept  have  potential,  a  pilot  with 
specific,  scientific  measurements  of  the  affects  of  same 
should  be  considered. 

3 .  Complexity  of  Data  Sources 

a.  Technology 

Research  Observation :  State  and  local  governments, 
typical  law  enforcement  agencies,  as  well  as  their  internal 
IT  service  providers  who  are  usually  responsible  for 
technology  initiatives  (such  as  the  topic  of  this  thesis) 
are  nearly  always  hamstrung  by  poor  financing,  antiquated 
technology/equipment ,  over-promised  (and  over-sold)  product 
capabilities,  and  most  importantly,  ill-defined 
requirements.  The  result  is  a  diluted  definition  of 
success.  Fundamentally,  these  trends  must  change;  first,  by 
taking  an  anthropological  approach  wherein  the  way  people  in 
a  fusion  center  actually  work  is  studied,  understood,  and 
respected,  and  ultimately,  ensuring  this  knowledge  is 
tightly  aligned  with  and  supportive  of  the  overall  mission 
of  the  organization.  This  type  of  contextual  solution 
requires  a  blended  team  of  experts  and  executive  level 
support . 
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Understanding  the  way  people  work  requires  first 
an  understanding  of  the  word,  "work."  The  work  of  an 
organization  is  the  aggregate  outcome  of  all  tasks 
undertaken  by  the  organization.  The  human  capital  (people) 
of  an  organization  execute  units  of  work  on  a  task  basis, 
possibly  utilizing  technology  to  enable  the  processes. 
Understanding  the  way  people  work  amounts  to  a  study  of 
tasks,  task  performance,  and  task  outcome.  In  other  words, 
people  are  at  the  center  of  the  equation. 

All  too  often,  consultants  converge  on  a  fusion 
center  at  the  beginning  of  a  new  project  usually  to  install 
and  configure  their  company's  solution.  These  consultants 
have  little  appreciation  for  law  enforcement  or  the 
intelligence  business  and  are  almost  exclusively  focused  on 
installing  "their"  product  which  then  places  the  burden  back 
on  the  client  to  ensure  the  product  is  effectively  applied 
to  their  work  challenge  and  the  project  is  properly  executed 
in  fulfilling  the  organization's  mission.  Inevitably,  money 
is  wasted,  and  equally  important,  missions  go  unfulfilled. 

Recommendation  #1  -  Understanding  and  Respecting 
the  Way  People  Work:  A  different  way  of  approaching  this 
type  of  problem  is  required;  in  essence,  a  solution  based 
upon  the  context  of  what  a  fusion  center  analyst  does.  It 
is  recommended  that  a  context-based  technology  solution 
specializing  in  assisting  humans  execute  fusion  center  tasks 
be  researched  and  developed. 

The  author  studied  one  such  potential  resolution, 
however,  is  not  about  to  propose  there  is  only  one  solution. 
This  recommendation  only  suggests  that  technology  be 
exploited  to  address  the  issue  of  information  overload. 
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complexity  of  data  sources  and  circular  reporting. 
Technology  works  24/7,  365  days  a  year,  assisting  each 
analyst  with  each  task.  Technology  is  never  fatigued  or 
stressed,  it  is  aware  of  all  available  data  sources,  it  is 
void  of  any  bias,  it  never  gets  distracted,  and  it  is 
unbound  in  its  ability  to  remember.  This  technology 
solution  should  assist  by  continuously  monitoring  the 
respective  situations  and  real  world  state  related  to  the 
assigned  task;  it  should  assist  by  suggesting  plans  and 
steps  based  upon  a  pre-determined  representation  of  Standard 
Operating  Procedures,  best  practices,  or  "plays;"  it  should 
assist  by  auto-executing  queries  to  needed  information  or 
notifications  to  other  entities  under  certain  conditions;  it 
should  assist  by  understanding  the  task  context,  the  intent 
of  the  human  analysts,  the  available  resources  (data  and 
people) ,  and  should  coordinate  these  based  upon  pre¬ 
determined  best  practices  embedded  within  the  technology. 

This  type  of  technology  would  allow  the  analyst  to 
focus  on  what  humans  do  best,  i.e.,  adapt,  learn,  recognize 
patterns,  perceive,  and  use  their  own  intuition,  without 
being  overloaded  by  gathering  and  understanding  the  inputs 
that  drive  these  unique  human  capabilities  and  qualities. 
In  other  words  the  technology  is  meant  to  help  the  human 
analysts  "make-sense"  out  of  a  very  specific  task  and  the 
situation  surrounding  it,  the  context,  so  that  the  analyst 
can  decide  what  and  how  to  proceed  to  a  successful  outcome. 

Recommendation  #2  -  Survey  of  Technologies:  As 
stated  in  the  main  thesis,  information  overload  is  not  a 
single  problem;  as  such,  it  will  probably  not  be  resolved 
with  a  single  solution.  Rather  the  solution  is  likely  to 
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come  from  synthesizing  multiple  components  into  a  single 
operational  system,  maximizing  the  aspects  of  each  in 
getting  relevant  information  that  is  organized  and  conducive 
to  the  current  task,  and  into  the  hands  of  the  analysts 
performing  the  task.  It  is  recommended  a  thorough 
assessment  of  current  products,  vendors,  and  research  be 
conducted  to  identify  viable  candidates  for  assembling  a 
context-based  technology  solution  for  fusion  centers. 

Recommendation  #3  -  Operational  Platform :  Given 
the  emphasis  of  this  recommendation  on  integrating  context- 
based  technology  into  the  heart  of  the  analysts''  work 
environment,  the  capability  of  the  work  environment  to  be 
integrated  with  it  cannot  be  overlooked.  The  ideal  work 
environment  would  be  flexible  enough  to  operate  in  centers 
with  different  missions,  open  enough  to  inter-operate  with  a 
variety  of  technology  and  analytic  products,  and  be 
customized  to  the  end  users',  fusion  center  analysts  needs. 
Like  the  other  items  in  this  list,  this  item  alone  will  not 
impact  the  information  overload  dilemma.  Only  when  this  and 
other  recommendations  are  implemented  will  these  solutions 
enable  making  real  headway  on  information  overload, 
specifically  the  issue  of  complex  data  sources. 

Jb.  Standard  Operating  Procedures  and  Training 

Research  Observation:  Before  attempting  to  apply 
technology  or  automation  to  a  problem  there  must  exist  an 
understanding  of  how  to  solve  the  problem  manually. 
Ideally,  the  manual  solution  will  be  reliable  and  repeatable 
by  the  human  resources  executing  the  procedure.  In  the 


fusion  center  business  Standard  Operating  Procedures  (SOP) 

are  known  steps  for  given  request  types  that  are  necessary 
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effectiveness. 


for  policy  compliance,  effectiveness,  and  efficiency. 
Ideally  these  procedures  embody  all  actions  and  techniques 
that  have  worked  in  the  past,  ensure  unsuccessful  techniques 
are  not  repeated,  are  continually  updated  and  refined,  and 
are  known  and  followed  by  all  center  staff. 


SOPs  exist  at  multiple  levels.  At  the  most 
qranular  level  of  detail  for  frequently  occurrinq, 
prescriptive  fusion  center  tasks,  such  as,  patrol  stop 
workups,  investigative  requests,  toll  analysis,  de- 
confliction,  etc.  could  and  should  be  documented  as  SOPs. 
At  a  more  general  level,  when  the  task  is  not  prescriptive, 
SOPs  could  guide  analysts  to  the  correct  data  sources 
respecting  the  circumstances  of  the  task. 


Unfortunately,  the  rapid  growth  of  fusion  centers, 
the  increasing  complexity  of  data  sources,  time  drains  of 
circular  reporting,  a  shortage  of  qualified  analytical  staff 
and  an  increasing  workload  have  resulted  in  most  center' s 
procedures  being  ad-hoc,  incomplete,  inconsistent,  and 
driven  by  individual  analysts'  capabilities,  knowledge,  and 
training . 


Recommendation:  An  adaptable,  automated, 
knowledge-driven,  best  practice  and  standard  operating 
procedures  technology  must  be  developed  and  accessible  to 
the  analysts.  It  is  through  these  automated  standard 
operating  procedures  that  technology  will  help  drive  the 
work  of  the  analysts.  Doing  so  will  minimize  the  complexity 
of  which  data  sources  should  or  should  not  be  utilized. 
Coupling  this  feature  with  automated  federated  access  to 
data  sources  will  further  reduce  the  negative  effects  caused 
by  the  complexity  of  data  sources.  When  analysts  are  able 
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to  make  a  single  query,  and  then  with  the  assistance  of 
enabling  technology,  the  "system"  extracts  relevant 
information  from  a  variety  of  data  sources,  the  business  of 
the  reduction  in  complexity  of  data  sources  will  begin. 

C .  FUTURE  RESEARCH 

The  phrase  "Information  Overload"  is  becoming 
commonplace.  Many  are  beginning  to  write  about  this  growing 
phenomenon.  What  is  absent  in  the  literature  is  a  focus  on 
a  multi-disciplinary  approach  with  an  intrinsic  focal  point 
on  the  end  user  and  their  cognitive  processes  while 
incorporating  the  social,  organizational,  contextual,  and 
technological  aspects  of  this  important  issue. 

Similarly,  the  issues  of  circular  reporting  and 
complexity  of  data  sources  has  not  been  thoroughly 
researched.  Noticeably  absent  from  the  literature  is  the 
causal  relationship  that  exists  between  circular  reporting, 
complexity  of  data  sources,  and  information  overload.  It  is 
certain  both  circular  reporting  and  the  complexity  of  data 


sources  will 

continue  to  plague 

the 

performance 

of 

intelligence 

units  throughout  the  world. 

The 

number 

of 

nodes  in  the 

network  amplifies  the 

problem  as 

well . 

As 

fusion  centers  and  intelligence  shops  continue  to  be  funded 
and  thereby  increase  in  number,  it  only  stands  to  reason 
that  the  prevalence  of  circular  reporting  will  only  be 
further  exacerbated. 

Secondly,  more  data  is  being  collected  than  in  the 
past,  and  that  trend  is  only  going  to  further  evolve.  In 
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addition,  those  data  sources  are  going  to  become  more 
prevalent  and  be  available  through  more  disparate  mediums 
than  ever  before. 

If  left  unaddressed,  it  is  clear  circular  reporting  and 
complexity  of  data  sources  are  going  to  continue  to  diminish 
the  capacity  of  fusion  centers.  These  issues  must  be 
researched,  understood  and  solutions  developed  in  order  to 
minimize  their  debilitating  effects  on  the  security  of  our 
communities  and  nation. 

Finally,  although  not  addressed  in  this  thesis,  a  key 
component  to  the  value  of  any  technology  is  how  easy  it  is 
for  the  user  to  use  the  technology  and  how  useful  they  feel 
the  system  will  be.  If  a  solution  is  not  easy  to  use  or  is 
not  perceived  to  be  useful,  it  will  be  underutilized; 
therefore,  it  is  imperative  these  two  factors  are  also 
subjects  of  future  consideration.  Although  the  findings  are 
clear  that  the  context-based  technology  researched  will 
increase  the  efficiency  and  effectiveness  of  fusion  centers, 
the  research  did  not  address  the  impressions  of  the  analysts 
regarding  ease  of  use  and  usefulness;  thus,  we  cannot  be 
certain  about  the  degree  of  use  for  the  system. 

D.  CONCLUSIONS 

The  utilization  of  a  context-based  technology,  tailored 
specifically  to  the  needs  and  mission  of  the  intelligence 
analysts  within  fusion  centers,  will  greatly  reduce 
information  overload  by  aiding  the  analyst  in  understanding 
and  approaching  each  task  based  on  the  context  of  that 
particular  task  and  that  analyst. 
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Embedding  the  context  of  both  the  task  and  the  analyst 
into  new  technology  will  directly  impact  the  business 
processes  and  operating  environment  of  a  fusion  center. 
This  will  also  reduce  the  information  overload  inherent  in 
the  increasingly  complex  data  and  information  sources. 
Overall,  the  effectiveness  and  efficiency  of  the  fusion 
center  will  be  dramatically  improved.  These  improvements 
will  come  by: 

•  Aiding  analysts  in  deciding  when  and  how  to  access 
and  utilize  the  multitude  of  information  sources 
available  to  a  fusion  center. 

•  Providing  for  more  efficient,  comprehensive  and 
consistent  task  performance  and  task  outcome  by 
having  the  context-based  technology  assist  and 
guide  the  human  analyst. 

•  Reducing  the  initial  training  time  for  analysts  to 
become  productive,  and  providing  continual 
training  as  the  knowledge  driving  the  decision 
support  system  is  updated. 

The  security  of  our  nation  and  even  smaller  communities 
rests  on  the  ability  of  public  servants  to  access, 
comprehend,  and  address  the  data  available  to  them.  It  is 
imperative  we  understand  the  challenges  these  servants  face 
and  then  develop  and  execute  viable  strategies  that 
facilitate  overcoming  the  daunting  challenges  they  face  in 
an  environment  where  this  explosion  of  information  overload 
is  only  expected  to  become  more  demanding  and  further 
exacerbated  by  circular  reporting  and  the  complexity  of  data 
sources . 
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APPENDIX  A  -  SURVEY  MAPPING 


Perceived  usefulness: 

1. 

Using  a  context-based  technology  product  in  my 
enable  me  to  accomplish  tasks  more  quickly. 

job  would 

2. 

Using  a  context-based  technology  product  will 
job  performance. 

decrease  my 

3. 

Using  a  context-based  technology  product  will 
productivity . 

increase  my 

4. 

Using  a  context-based  technology  product  will 
effectiveness  on  the  job. 

decrease  my 

5. 

Using  a  context-based  technology  product  will 
difficult  to  do  my  job. 

make  it  more 

6. 

I  will  find  a  context-based  technology  product 
j  ob . 

useful  in  my 

Perceived  ease  of  use: 

1.  Learning  to  utilize  a  context-based  technology  would  be  easy 
for  me. 


2.  I  have  found  it  easy  to  use  a  context-based  technology  to 
obtain  decision-making  material. 

3.  Using  a  context-based  technology  is  complicated. 

4.  I  have  found  a  context-based  technology  to  be  inflexible  to 
interact  with. 


5.  It  would  be  easy  for  me  to  become  skilled  at  using  a 
context-based  technology. 

6.  I  have  found  the  context-based  technology  product  difficult 
to  use. 

Perceived  effect  on  Information  Overload: 

1.  The  use  of  a  context-based  technology  will  reduce  the  time  I 
spend  collecting  information. 

2.  The  use  of  a  context-based  technology  will  increase  the  time 
I  spend  collating  information. 
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3.  The  use  of  a  context-based  technology  will  decrease  the  time 
I  spend  collating  information. 

4.  The  use  of  a  context-based  technology  will  improve  my 
ability  to  handle  the  information  available  to  me. 

5.  The  amount  of  information  I  am  expected  to  review  makes  it 
difficult  for  me  to  complete  strategic  analysis. 

6.  The  amount  of  information  available  to  me  is  difficult  to 
handle . 

Perceived  effect  on  Complexity  of  Data  Sources: 

1.  It  is  difficult  to  access  the  data  sources  available  to  me. 

2.  The  number  of  data  sources  I  am  required  to  check  makes  it 
difficult  for  me  to  provide  a  timely  response. 

3.  The  number  of  data  sources  available  to  me  is  easy  to 
manage . 

4.  It  would  be  beneficial  to  me  if  more  data  sources  were 
accessible  through  a  context-based  technology. 

5.  Using  a  context-based  technology  will  make  it  more  difficult 
to  handle  the  number  of  data  sources  available  to  me. 

6.  Using  a  context-based  technology  will  make  it  easier  to 
handle  the  number  of  data  sources  available  to  me. 

Perceived  effect  on  Circular  Reporting: 

1.  The  use  of  a  context-based  technology  will  reduce  the  number 
of  times  I  am  required  to  read  duplicate  information. 

2.  The  use  of  a  context-based  technology  will  increase  the 
number  of  times  I  am  required  to  read  duplicate  information. 

3.  Frequently  I  am  expected  to  read  duplicate  information. 

4.  Seldom  am  I  expected  to  read  duplicate  information. 

5.  My  productivity  is  negatively  impacted  by  the  amount  of 
times  I  am  expected  to  read  duplicate  material. 

6.  There  are  occasions  when  I  put  together  an  intelligence 
product  and  later  see  it  being  reported  back  to  me. 
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APPENDIX  B 


SURVEY  INSTRUCTIONS 


Your  participation  in  the  survey  is  completely 
vol  un  tary . 

The  following  outlines  the  process  for  completing  the 
survey.  These  steps  have  been  put  into  place  to  assure  you 
that  your  responses  will  remain  anonymous. 

All  criminal  intelligence  analysts  who  are  assigned  to 
the  Statewide  Terrorism  and  Intelligence  Center,  regardless 
of  your  parent  agency,  will  be  given  the  opportunity  to 
participate.  If  you  choose  not  to  participate,  that 
decision  will  not  be  made  known  to  the  research  assistants, 
the  graduate  student  or  your  parent  agency. 

At  the  beginning  of  the  survey  you  will  be  asked  to 
identify  whether  you  have  less  than  three  years  of 
experience  as  an  intelligence  analyst  or  if  you  have  more 
than  three  years  of  experience  as  an  intelligence  analyst. 
Based  on  what  we  know  about  the  potential  respondents,  we 
believe  half  have  less  than  three  years  of  experience  and 
half  have  more  than  three  years  of  experience. 

If  you  choose  to  participate  you  will  read  30 
statements,  you  are  asked  to  circle  the  answer  which  most 
closely  represents  your  perceptions  of  the  statement. 
Possible  responses  will  vary  between  Strongly  Agree  and 
Strongly  Disagree. 

You  will  be  greeted  by  a  research  assistant  and  asked 
to  complete  the  survey  in  a  private  office.  Once  finished 
you  will  leave  that  office  and  enter  a  second  private 
office.  You  will  be  greeted  by  a  second  research  assistant. 
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At  the  time  the  second  research  assistant  will  ask  you  if 
you  have  any  questions  or  if  any  of  the  statements  were  not 
understood.  Once  you  are  comfortable  with  the  responses  you 
will  place  the  survey  in  an  envelope,  seal  the  envelope,  and 
place  it  in  a  sealed  box. 

The  sealed  box  with  the  completed  surveys  will  not  be 
opened  until  all  34  respondents  have  had  the  opportunity  to 
complete  the  survey  and  place  it  in  the  sealed  box. 

If  you  elect  not  to  participate,  you  will  go  through 
the  same  process.  This  will  keep  the  research  assistants 
from  knowing  who  participated  and  who  did  not. 

There  are  no  statements  on  the  survey  that  contain 
information  that  can  be  traced  to  any  individual. 

A  graduate  student  prepared  the  attached  survey.  The 
information  obtained  from  the  survey  will  be  compiled  and 
included  in  the  students'  graduate  work.  In  addition  to  the 
completed  work  being  publicly  accessible,  it  will  also  be 
distributed  directly  to  the  following  entities: 

Department  of  Homeland  Security 

Drug  Enforcement  Administration 

Federal  Bureau  of  Investigation 

Illinois  National  Guard 

Illinois  State  Police 

Illinois  Terrorism  Task  Force 

Naval  Postgraduate  School 

RiverGlass,  Inc. 

Statewide  Terrorism  and  Intelligence  Center 
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We  thank  you  for  your  participation. 

I  have  read  the  instructions  above  and  I  fully 
understand  that  my  participation  is  completely  voluntary.  I 
have  no  further  questions  and  am  prepared  to  continue. 


Signature 

Date 
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APPENDIX  C 


SURVEY 


Throughout  the  attached  survey  you  will  find  the  term 
context-based  technology.  In  an  effort  to  insure  all  of  the 
respondents  fully  understand  the  author' s  use  of  this  term; 
the  following  paragraph  is  used  to  describe  the  intended 
meaning . 

Most  attempts  to  solve  the  information  overload  problem 
have  neglected  the  most  critical  pieces  of  any  solution:  the 
people,  the  nature  of  their  work,  their  goals,  their  roles, 
their  organizations,  their  experience,  and  finally  their 
training;  in  other  words,  the  "context"  of  the  user.  In  a 
highly  technical  field  this  is  not  unusual.  Technology 
developed  around  a  context-based  approach  inverts  the  usual 
methodology.  Instead  of  starting  with  the  technology,  and 
naively  believing  all  problems  will  be  solved  if  everyone 
could  retrieve  and  share  information  with  others,  the 
context-based  approach  starts  with  the  user  and  the  user' s 
tasks,  roles,  and  goals.46  The  contextual  approach  then 
dictates  what  type  of  analytics  and  visualizations  need  to 
be  brought  to  bear  to  create  an  effective  solution  to  a 
particular  problem. 

Within  the  Statewide  Terrorism  and  Intelligence  Center, 
a  new  system  has  recently  been  introduced,  RiverGlass. 


46  Keith  Jarvelin,  Peter  Ingerwesen  and  Nick  Belkin,  "Information 
Retrieval  in  Contexts"  (SIGIR  2004  IRiX  Workshop,  Sheffield,  England, 
July  29,  2004) . 

http : //ir . dcs .gla.ac.uk/ context/ I RinCon text  WorkshopNotes  SIGIR2004 . pdf 
(accessed  March  17,  2007) . 
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RiverGlass  has  been  designed  from  the  intelligence  analyst's 
perspective.  Therefore  it  is  said  to  be  a  context-based 
technology . 

As  you  complete  the  attached  survey  you  should  use 
RiverGlass  as  your  frame  of  reference. 
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Using  a  context-based  technology  product  will  decrease 


my  job  performance. 

Strongly  Agree 
Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


1.  I  have  found  a  context-based  technology  product  to  be  inflexible 
to  interact  with. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


2.  The  use  of  a  context-based  technology  product  will  increase  the 
time  I  spend  collating  information. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 
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3.  The  use  of  a  context-based  technology  product  will  improve  my 
ability  to  handle  the  information  available  to  me. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


4.  There  are  occasions  when  I  put  together  an  intelligence  product 
and  later  see  it  being  reported  back  to  me. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 

5.  The  use  of  a  context-based  technology  product  will  increase  the 
number  of  times  I  am  required  to  read  duplicate  information. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 
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6.  Using  a  context-based  technology  product  will  make  it  more 

difficult  to  handle  the  number  of  data  sources  available  to  me. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


7.  The  amount  of  information  available  to  me  is  difficult  to 
handle . 

Strongly  Agree 
Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


8.  Using  a  context-based  technology  product  will  make  it  more 
difficult  to  do  my  job. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 
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9.  My  productivity  is  negatively  impacted  by  the  amount  of  times  I 
am  expected  to  read  duplicate  material. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


10. The  use  of  a  context-based  technology  product  will  decrease  the 
time  I  spend  collating  information. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


11. Using  a  context-based  technology  product  will  decrease  my 
effectiveness  on  the  job. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 
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12. Learning  to  utilize  a  context-based  technology  product  would  be 
easy  for  me. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


13.1  have  found  a  context-based  technology  product  difficult  to 
use . 

Strongly  Agree 
Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


14. The  number  of  data  sources  I  am  required  to  check  makes  it 
difficult  for  me  to  provide  a  timely  response 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 
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15. Using  a  context-based  technology  product  will  make  it  easier  to 
handle  the  number  of  data  sources  available  to  me. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


16. Seldom  am  I  expected  to  read  duplicate  information. 
Strongly  Agree 
Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


17.1  will  find  a  context-based  technology  product  useful  in  my  job. 
Strongly  Agree 
Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 
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18. Using  a  context-based  technology  product  is  complicated. 
Strongly  Agree 
Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


19. Using  a  context-based  technology  product  in  my  job  would  enable 
me  to  accomplish  tasks  more  quickly. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


20. It  would  be  beneficial  to  me  if  more  data  sources  were 
accessible  through  a  context-based  technology  product. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 
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2 1 . Frequently  I  am  expected  to  read  duplicate  information. 
Strongly  Agree 
Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


22. The  use  of  a  context-based  technology  product  will  reduce  the 
number  of  times  I  am  required  to  read  duplicate  information. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


23. Using  a  context-based  technology  product  will  increase  my 
productivity . 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 
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24. It  would  be  easy  for  me  to  become  skilled  at  using  a  context- 
based  technology  product. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


25. The  amount  of  information  I  am  expected  to  review  makes  it 
difficult  for  me  to  complete  strategic  analysis. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


26. It  is  difficult  to  access  the  data  sources  available  to  me. 
Strongly  Agree 
Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 
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27. The  use  of  a  context-based  technology  product  will  reduce  the 
time  I  spend  collecting  information 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


28.  I  have  found  it  easy  to  use  a  context-based  technology  product 
to  obtain  decision  making  material. 

Strongly  Agree 

Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


29. The  number  of  data  sources  available  to  me  is  easy  to  manage. 
Strongly  Agree 
Agree 

Somewhat  Agree 
Undecided 
Somewhat  Disagree 
Disagree 

Strongly  Disagree 


31.1  have  more  than  3  years  of  experience  as  a  criminal 
intelligence  analyst.  Yes  No 
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